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No Agape for Latin 

Agape is not Latin. It is Greek. What does 
this say about the rest of his article? [See 
Jon “maddog” Hall's Beachhead in the June 
2007 issue of Linux Journal. 


~— 


Asynchronous Data Access 


Tobin 


Jon “maddog” Hall’s heart was in the right 
place, but his research was defective. Agape 
is Greek, not Latin. | submit this small criti- 
cism in the highest sense of the word. 


Daryl 


A note for Jon “maddog” Hall: | was always 
taught (and Wikipedia confirms) that agape 
is of Greek origin, not Latin. Otherwise, it 
was an interesting stroll down memory lane. 
What gives? 


John E. Young 


Maddog, | enjoyed your article, but | almost 
didn't make it past the second paragraph; 
agape is a transliteration from Greek not 
Latin—a compiler error | presume. 


| can assure you assembly language is still 
kicking—the last assembly language (IBM 
360 type, now 2/Arch type) program | wrote 
was yesterday. 


Richard Pace 
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One can only hope that Jon “maddog” 
Hall learned his programming languages 
better than he learned his Latin. In his 
June 2007 column, he says that he named 
his boat Agape from the Latin word mean- 
ing “the highest form of love”. The word 
agape is not Latin, as Hall suggests, but 
Greek, and being one of several Classical 
Greek words for different types of love, it 
generally is understood to mean some- 
thing like “brotherly love” or charity. 
Maybe maddog needs to debug his Latin. 


Max E. Klinger 


The origin of the word agape is Greek, 
but Latin assimilated it. As Jon’s teacher 
taught Latin, not Greek, she was correct 
in teaching agape as a Latin word. One 
other note: agape is often translated as 
charity, but philia is the Greek for brotherly 
love.—Ed. 


More FORTRAN before Interface 
A heartfelt thanks to Jon “maddog” Hall 
for both his article on languages in the 
June 2007 issue and his dedication of that 
issue to John Backus. While my career 
diverges wildly on details from Jon's, there 
was enough commonality to draw a hearty 
“Amen, Brother Jon” at the end of his 
column. | also started with FORTRAN II 

(on CDC Big Iron) in the late 1960s as an 
astronomy student, and was drawn into a 
computer systems class that was mostly a 
class on CDC's assembly language and 
CDC computer architecture. Knowing how 
all those Os and 1s work together to get 
all that hardware to do what you want is 
definitely an essential knowledge to any- 
one writing serious code. I’ve not written 
anything at the machine level in decades, 
but, as Jon said, what | learned in pro- 
gramming “kindergarten” is in the back of 
my mind any time | write any code, be it 
in FORTAN (still my primary language), 
Java, PostScript, Python or whatever lan- 
guage | need to use. 


Jim Secan 


Never Fight a Land War in Asia 
First off, love the magazine. | look forward 
to each issue even with a large portion of 
it being over my head. 


| often spend time telling people of the 
merits of Linux and that we are winning— 
much like Doc Searls’ “Picking New 
Fights” article in the June 2007 issue. But 
| never fail to run into the standard, 
“what about obscure hardware or newer 
hardware that doesn’t run on Linux? With 
Windows, it just works.” 


| then have to explain that, no, it doesn’t 
“just work". The manufacturers simply 
write drivers under Windows, and if they 
would get pressure to write for Linux, it 
would “just work” there as well. Our 
answer is that the manufacturers are the 
ones that should spend time testing for 
compatibility. That would, in turn, also 
free up Linux programmers to write other 
useful stuff. 


Nick Petreley, in /var/opinion [June 2007], 
fell victim to one of the classic blunders. 
The first of which is never get involved in 
a land war in Asia. The second, which is 
only slightly less known, is never go in 
against a Sicilian when death is on the 
line, and the third (even slightlier less 
known) is always blame the manufacturer 
for incompatibility. 


We blame ATI for crummy Linux drivers, 
and not Linux itself, right? ATI, the manu- 
facturer, we say, should be responsible for 
basic support and contribution to the 
Linux community to support its hardware, 
correct? So why doesn’t jEdit create an 
Ubuntu package and submit it? 


| realize it wasn't a specific complaint 
against Ubuntu, but the same assumption 
is there—that it’s Ubuntu’s responsibility to 
go out and do the legwork. Better yet, 
how about Nick making an Ubuntu pack- 
age and submitting it? 


You guys are great. | just wanted to share 
with you my initial impression. | know that 
one day everything will hopefully just gel 
together. If you have a program you like, 
ask to make it available for your favorite 
distro, etc. 


Kermit Jones 


| love the references to Princess Bride, 
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thanks. If | had time to create packages 
for Ubuntu, | would.—Ed. 


To Doc from Doc 
| just read your June 2007 column and 


felt compelled to write about a few issues. 


[See Doc Searls’ “Picking New Fights”. ] 


lam a Chiropractor, running a small 
office in Sandy, Utah. Perhaps | have 
overlooked the buzz of Linux enthusi- 
asts coding for healthcare projects. Any 
ideas on this? 


Another area | find lacking...is small 
business. | am a regular in several chan- 
nels on irc.freenode.net as bonez39, in 
#utah and #debian. | have many friends 
here, who are talented coders, well 
versed in Linux and a host of tools and 
anguages. Where it becomes disheart- 
ening for me, running a small business, 
is when | seek out other small-business 
owners, in such channels. 


t seems that | find only programmer/coder 
ypes, and further, it seems that people 
running small businesses are too busy run- 
ning the business to mingle in such 
groups. Do you know of any groups 
devoted to small-business owners, who 
also hope to run things efficiently and 
effectively with Linux? 


Amen to the fight against the telcos. 

| recently switched over at my office 
from Qwest, where 1.5 was the 
maximum rate for broadband, up to 
4+Mb with Comcast. This is at the base 
business rate, so | can keep my over- 
head down. 


What can I, as one business owner, do 
to motivate patients, friends and family 
to further the open mobile and free- 
dom movement with telcos? 


You've got a new devoted reader here! 
Thanks for a great piece this month. 


Dr. Scott S. Jones 


| Never Metaprogramming 

| Didn't Like 

| was really pleased to see the article on 
metaprogramming in the June 2007 
issue. [See Ariel Ortiz’s “An Introduction 
to Metaprogramming”.] After more 
than a decade of writing Fortran pro- 
grams that write PostScript programs, 
and shell scripts that write Fortran, and 
awk scripts that make the PostScript 
frames that a shell script turns into a 
GIF animation, | finally came across the 
word metaprogramming a couple of 
years ago. Certainly a major motivation 
or this kind of work is to make the 
machine produce the boilerplate for 
some programming language. | liked 
he article, but | was surprised to see 
bash ignored, except for an oblique 
mention that “Some languages have a 
acility called eval.” 


Finally, after all the interesting articles, 
here's Nick Petreley’s opinion column. 
Is it just me, or has LJ become a lot 
more interesting since he took the 
helm? Thanks to you all, in any case. 


Andrew T. Young 


Who Owns OpenGL? 

In the editor’s comment to the “Miniature 
OpenGL Development System” letter in 
the June 2007 issue, you claim that 
“Microsoft owns the patent to OpenGL”. 


That is not true, and it’s a bad example of 
spreading FUD that | wouldn't have 
expected from you. 


Microsoft bought some patents from 
Silicon Graphics in 2002 that may be 
applicable to parts of OpenGL. (And very 
little has ever surfaced again, but then 
again, with patents you never know.) 


Ulrich Hertlein 


| sit partially corrected. Microsoft has 
staked claims on important portions of 
OpenGL, not the entire thing.—Ed. 
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Linus Torvalds recently 
said that the whole 
software suspend 


® 
diff -u 
WHAT’S NEW debate between Pavel 
IN KERNEL Machek’s swsusp and 
DEVELOPMENT Nigel Cunningham's 
suspend2 was entirely 
misguided. Linus said he no longer has faith 
that either project can succeed. His hope now, 
he said, was that some new person would 
step in and take the whole concept in an 
entirely new direction. He also said that he 
believes software suspend is indeed a kernel 
issue and not something that should be entirely 
in user space. If he finds a solution he likes, 
he'll incorporate it into the kernel. 
also seems that Linus himself has solved 
some of the basic development issues, and 
various people are now telling him his ideas 
are crazy. Typically when this happens, folks 
will hurl insults at Linus for weeks or months, 
while he'll just continue to explain his ideas. 
Eventually, some kind of tipping point is 
reached, and a general perspective shift 
occurs. At this point, one or more folks will 
rush off to code up the new idea. 

Alan Hourihane of Tungsten Graphics 
has been working on an FBDev driver for 
Intel’s LE80578 chipset. Intel actually had 
been funding the work through Tungsten, 
and Alan posted his code recently to the 
linux-kernel mailing list. In response to vari- 
ous technical suggestions, Alan also posted 
updated patches addressing those issues— 
typically a sign that a piece of code is likely 
to be included in the official kernel sooner 
rather than later. 

The question of what will become of the 
Reiser4 filesystem is still open. It could be 
adopted into the official kernel; it could trans- 
form from a primarily corporate project to a 
primarily volunteer one; it could become 
unmaintained and disappear completely. 
There are many possibilities. But, the fact of 
Hans Reiser’s murder trial does not change 
most of the issues surrounding whether to 
include the code in the kernel. 

One thing holding the project back is that 
kernel developers are still reluctant to give sig- 
nificant feedback on the code. Hans’ habit of 
rejecting all commentary and engaging in ad 
hominem attacks on the commentators has 
left a mark that even his absence on murder 
charges has not eroded. A lot of folks who 
had put in a significant amount of time on 
giving analysis of Reiser4 code simply have 
moved on to other projects, and no one has 
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arisen to take their place. 
Without reviewers, the likelihood of Reiser4 
going into the kernel diminishes greatly. A num- 


ber of relatively large technical issues are standing 


in the way of inclusion, and reviews by kernel 
developers were one of the key ways the Reiser4 
developers could understand those issues. 

It seems possible that Andrew Morton 
might try to jump-start some kind of review 
process by accepting the patches into his - 
mm tree, with the implied threat/promise of 
submitting them to Linus Torvalds. This would 
certainly have a number of kernel developers 
up in arms and cause them to take a much 
closer look at the code. But, it also is a fairly 
drastic measure for Andrew to take. 

It's also possible that any sort of review 
process might take too long. The Namesys 
engineers who maintain Reiser4 are having 
to confront a diminishing amount of funds 
available to support the project. At least one 
Namesys engineer, Edward Shishkin, has 
said he would still devote about 25% of his 
time to Reiser4 on a volunteer basis if his 
employment at Namesys went away. 

As usually happens when issues like these 
are brought out in the open, recent discussions 
on linux-kernel have resulted in several folks 
showing interest in reviewing the Reiser4 
code. None of the reviewers from the old days 
seem very interested, but maybe this new set 
of reviewers will be able to provide enough 
commentary to smooth the path a little bit. 
Adrian Bunk continues his long-standing 
efforts to clean up the kernel. A lot of kernel 
code hangs around in the kernel for a long 
time, even after a better alternative comes 
along, and even when that better alternative 
makes its way into the official source tree. 

In the current case, X86_ACPI_CPUFREQ 
is the new code, allowing users to change the 
clock speed of their CPUs on the fly, which 
can amount to a large power savings. The old 
code, X86_SPEEDSTEP_CENTRINO_ACPI, has 
been deprecated for a long time, and now its 
time has come. 


Adrian posted a patch removing the older 
code, and there was a small bit of discussion— 
whether to include the patch in 2.6.21 or 2.6.22, 
and how to communicate the change to users 
who might rely on the older code. Bill Davidsen 
was particularly worried about this latter possibili- 
ty, until Adrian reminded him that the older code 
had been marked as deprecated for a long time, 
and the feature-removal-schedule.txt file con- 
tained a clear explanation of the replacement. 

—ZACK BROWN 


LJ Index, 
August 2007 


1. Position of Linux-based Rackspace among 
Netcraft’s most reliable hosting companies 
for April 2007: 1 


Number of Linux-based hosting providers 
among Netcraft’s top 10 for April 2007: 3 


Number of open-source hosting providers 
among Netcraft’s top 10 for April 2007: 6 


Millions of dollars in 2006 revenues for 
Rackspace: 224 


Rackspace revenue growth percentage over 
2005: 61 


Number of open-source venture capital deals 
in 2004: 36 


Millions of dollars invested by VCs in 
open-source startups in 2004: 297 


8. Number of open-source venture capital deals 
in 2005: 41 


9. Millions of dollars invested by VCs in 
open-source startups in 2005: 306 


10. Number of open-source venture capital deals 
in 2006: 48 


11. Millions of dollars invested by VCs in 
open-source startups in 2006: 481 


12. Billions of dollars invested by VCs in 
Open-source startups since 2000: 1.9 


13. Number of five-year road maps for the Linux 
kernel: 0 


14. Number of operating systems that support 
more hardware than Linux: 0 


20. Year by which the amount of data to be 
stored exceeds the capacity of data storage 
devices: 2007 


1-5: Netcraft.com 
6-12, Matt Asay, Robin Vasan and Matthew Aslett 
13, 14: Jonathan Corbett 
15-20: IDC, via Freedom’s Phoenix 


—Doc Searls 
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Here’s a Hitch 


The Linux Foundation now has a travel 
fund for developers who need to 
attend technical conferences but 


Collaboration Summits, the Linux 
Foundation’s Japan Symposiums, the 
Kernel Summit, Ottawa Linux 


=— 


can't make it on their own. pe Symposium, Linux.conf.au, 
The fund is open to LR a desktop confer- 


“elite community 
developers with a 
proven track 
record of 
open-source 
development 
achievements 
and who 
don’t 
otherwise 
have access 
to funding for 
attending technical 
events”. In other 
words, it's “for the 
rock stars of the open- 
source world”, as Jim Zemlin, 
Executive Director of the Linux 
Foundation, puts it. 

The fund covers travel to “LF 


: ences, such 
as Guadec 
and 
Akademy, 
and other 
technical 
conferences 
where true 
collaboration 
takes place”. 
More pointedly, 
it “does not 
offer sponsorships 
Yo tradeshows or 
nontechnical 
conferences”. 
If this is for you, or if you wish it 


were, visit www.linux-foundation.org/ 
en/Travelfund. 


—DOC SEARLS 


Dude! It’s an 


Ubuntu! 


Last month, we visited the overwhelming pro-Linux 
response to Dell's IdeaStorm site (“New Product 
Design: Now You Decide!”, www.ideastorm.com). 
As we go to press, there is much talk about Dell’s 
plans to market PCs with Ubuntu pre-installed. That 
talk has been fueled by the biography page of 
company founder and namesake Michael Dell. At its 
top, above all the other computers he uses, is this: 


What toed of computer Goes Mches! Det tave? See tor yourset These are De 
FytteTS 8nd Deroreres Mictae 8 UENg NOY TOW M you wart i wer more 
about Mese products. or purchase one of your oar. val Ine ivan 
At home: Dell Precision M90 
Hardware: 
© Wee Core 2 Quo T7600 
Processor 
(© 460 COR? 667MNz DRAM 
© 171 WHGA+ Weescreen 


Software 
© Ubuntu 7.04 Featy Fawn 
© VMware Workstation 6 


Gee 
© OpenOtics org 22 
© Atomatix? 
© Fretox 2003 
© Evoktion Grourmare 2 10 
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JavaFX Mobile: It Only Starts with the Phones 


First there was the Qtopia Greenphone 
(www.trolltech.com/products/qtopia/ 
greenphone) from Trolltech: “a Linux mobile 
development device open for unlimited soft- 
ware innovation”. Then there was OpenMoko 
(wiki.openmoko.org/wiki/Introduction): 
“the world’s first integrated open-source 
mobile communications platform”. 

Now there's JavaFX Mobile: “a complete 
mobile operating and application environment 
built around Java and Linux open-source tech- 
nologies”. Not coincidentally, the phone Sun 
used to demonstrate JavaFX Mobile at 
JavaOne in May 2007 was the OpenMoko 
design (known as Neo1973) that we covered 
in the February 2007 issue of Linux Journal. 

Sun and Java have been around the cell- 
phone business for a long time. And, the 
platform has come a long way since the time 
| joked to Sun CEO Jonathan Schwartz that 
Java to me was “the logo | had to stare at 
for 16 seconds after | hit the wrong button 
on my cell phone”. 

But, this is a huge new move, for five rea- 
sons. First, Java is being GPL'd (which was the 
cover story of our June 2007 issue). Second, 
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Sun wants to fill the world with mobile 


phones that are open to development by play- 


ers other than the phone builders and their 
carrier partners. Third, Sun actually has clout 
with phone makers. Fourth, it's not stopping 
at phones, but looking to put this in all kinds 
of embedded devices. Fifth, but hardly last, 
Schwartz & Co. have a bone to pick with 
Steve Jobs, who has dissed Java while getting 
huge PR for Apple’s new (and closed) iPhone. 

There are lots of dots to connect here. 

There's SavaJe, the source of the code 
base for JavaFX Mobile, recently acquired by 
Sun. There are connections between SavaJe 
and longtime Linux hardware developer 
Tadpole Technology. 

There's Ed Zander, the Motorola CEO 
who used to be Sun's President and CEO, 
and who can’t be happy to have seen 
Motorola’s phone “partnership” with Apple 
go south. Would Motorola be willing to go 
zig in the open direction while Apple zags in 
the closed? Good chance. 

There's Eric Schmidt, the former Java 
development chief at Sun who is now CEO 
of Google. He’s on Apple's board, but still, 


it’s a connection. 

Then, there’s what Jonathan Schwartz 
says on his blog (blogs.sun.com/jonathan/ 
entry/when_not_where): he wants the 
world to have an open platform that allows 
“any consumer electronics manufacturer to 
accelerate the delivery of Java/Linux-based 
devices, from phones to set tops and dash- 
boards and everything else imaginable, with- 
out fear of format lock-in or disintermediation 
from a competitor”. All this from “a company 
with no agenda to disintermediate content 
owners and every interest in propelling the 
Open Source community (every portion of the 
content Sun contributes to the JavaFX product 
and community will be via the GPL license, at 
the core of Java and GNU/Linux)”. 

He also wants this to make phones 
cheap as well as open, to “play a central role 
in bringing the Internet to the planet” and 
to “be the software to build the devices to 
bridge the digital divide”—like the One 
Laptop Per Child, only half the price or less. 

Encouraging stuff. It'll be interesting to 
see what happens. 


—DOC SEARLS 
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inside™ 


Universal I/O 


Maximized I/O Expandability & Flexibility 


The 
Power of 7} 


Three Add-On ee 


Gives you the freedom to build 
exactly what you need! 


Supermicro UIO servers allow users to select from a 
wide range of I/O options to provide the ultimate in 
storage and networking flexibility. The UIO card 
becomes a part of the serverboard, allowing the system 
to retain all of it’s PCI Express and PCI X slots for 
expansion cards. As a result, future upgrades can be Optimized Chassis Solutions 
achieved by replacing the UIO card and/or expansion 
cards instead of replacing the entire system. This 
versatility helps to minimize the amount of different 
servers that customers need to operate their business. 


._*** 


SC815TQ-R650U SC825TQ-R700U 
V 3/7 Add-on Cards for 1U/2U 


V Highly Upgradeable 


UIO Motherboard Options 


V High Efficiency Power (up to 90%+) 
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(SAS RAID 5, 10-G, IB...up to 20 choices) 
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1-800-800-6328 1-888-427-2250 1-800-2000-ASI 1-800-232-9920 1-800-456-8000 1-408-941-0808 1-800-756-5974 1-800-237-8931 
www.amax.com www.arrownacp.com www.asipartner.com www.bellmicro.com www.ingrammicro.com www.malabs.com www.synnex.com www.techdata.com 


© 2007 Super Micro Computer, Inc. Specifications subject to change without notice. All other brands and names are the property of their respective owners 
Intel, the Intel logo, Intel inside, the Intel Inside logo, Intel Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. 


( UPFRONT | 


Thin Clients Rock 


hen you introduce Linux to a group of 

people unfamiliar with the idea, it's a 
ot like giving cough syrup to a five-year-old. 
Average users tend to lose sight of the 
hings Linux can do for them and focus on 
he rough edges. As Linux “people”, our job 
is often just as political as it is technical, 
because let's face it, the software is generally 
he easiest part of the equation. 

In 2003, Michigan's educational fund- 
ing started a downward trend that contin- 
ues into present day (2007). Every school 
in our state feels it, and every school is 
competing to keep students. In an unfor- 
unate twist, 2003 also was the year we 
needed to change our student manage- 
ment system from our old, now defunct, 
acintosh-based system to a Microsoft 
Windows-based solution. My suggested 
method to accomplish that was Linux— 
specifically, Linux thin clients. 

| created a menu of proposals for the 
school board. One described the cost to 
switch entirely to a Microsoft-based infras- 
tructure. This included workstations, 
servers, licensing, staff training and addi- 
tional third-party needs, such as antivirus 
software and Microsoft Office. The other 
option required purchasing a handful of 
servers and using donated computers as 
thin clients. We would base the system on 
the Linux Terminal Server Project (LTSP). 
The first proposal came in at about 
$500,000 for the first year, and the latter 
came in at about $150,000. | didn’t have 
to do much more selling—" Linux” was 
the school board's new favorite word. 

It's important to note that | easily 
could have quoted the cost at $50,000 
and still had a functional network. One of 
he biggest mistakes we as OSS advocates 
make is not spending enough money. 
Don't scrimp on hardware when you are 
saving so much on software. It’s important 
o find a balance between savings and 
unctionality. For example, the bare mini- 
mum | needed to convert our district to 
hin-client technology was: 


™ A fast LTSP server per 50-75 thin clients 
to handle the booting. 


™® A Windows Terminal Server and appropri- 
ate licensing. 


® Actual thin clients (we were a Macintosh 
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school, and at that point, old Mac hard- 
ware couldn't be used for thin clients, so 
we had to acquire new ones). 


However, my quote also included the fol- 
lowing additional purchases: 


™ Two more fast servers to support addi- 
tional thin clients and/or load balancing. 


™® A complete network upgrade (switches 
and so on) to a 1Gb backbone. 


® Microsoft Office licenses to ease the 
transition to OSS for naysayers. 


™ New 17" monitors instead of recycled 
15" monitors. 


The board approved the transition 
to Linux. We ordered the hardware and 
jumped feet first into an exciting summer. 

LTSP is a system that works on top of 
basically any Linux distribution. I’ve always 
been rather fond of Debian, but for this 
huge install, | wanted to use something 
that had been tried and tested. The folks 
over at K12LTSP have taken Fedora and 
created an installer that integrates LTSP 
and scores of tweaks that really make the 
thin clients shine. The support from their 
mailing list is usually quick and always 
provides more help than any phone sup- 
port I've ever used. In fact, the developers 
usually are available in real time on IRC 
if you have a mission-critical question. 
Unless you have a real aversion to Fedora- 
based distributions, there’s really not any 
good reason to go elsewhere. 

One of the more difficult parts of 
implementing a wide-scale thin-client roll- 
out is to estimate server requirements. 
There is no simple way to determine how 
many thin clients a given server can han- 
dle. The requirements vary enormously 
depending on the applications that will 
run from the thin clients. A best guideline, 
based on discussion with the LTSP devel- 
opers, is the following: 


™ Get the fastest processors you can afford. 
Dual/quad is better than single. 


™ Hard drives, as long as file storage is 
done remotely, really don't have to be 
super fast. 


™ Reserve 512MB of RAM for the server 
and 100MB per thin client. 


™ 50 clients is a lot on a server. For more 
than that, consider multiple servers. 


® Gigabit Ethernet on a switched network is 
a must. Dual gigabit is better. 


@ File serving for a large network is best 
done from a separate NFS server, which 
has the fastest drives you can afford. 


Following those guidelines, | figured in 
order to support the 75 thin clients | need- 
ed, | should get three LTSP servers, one file 
server and one Windows Terminal Server 
(for the Windows application we needed). 

During the installation, one of my 
iggest worries was not having enough 
orsepower to run all the thin clients. The 
first concern was network bandwidth. We 
did purchase new switches throughout the 
istrict, but | wasn’t familiar with Ethernet 
channel bonding, and | didn't want to 
experiment with the idea on the first day 
of school. | decided to split the network 
traffic into two segments. All traffic 
between the LTSP servers and the thin 
lients traveled over ethO on our main 
network, and all file transfers between the 
LTSP servers and the NFS server flowed 
through a separate, gigabit switch over 
eth1. That effectively isolated the two 
main bandwidth usages and avoided the 
need for channel bonding. The solution 
was very effective. 

| was still concerned about the perfor- 
mance issue and didn’t want my igno- 
rance to make LTSP or Linux look bad 
on day one. To make sure bandwidth and 
processor usage wouldn't be a problem, 
| decided to give the users a very spartan 
desktop when they logged in. The soft- 
ware | chose for a minimalistic Linux 
experience was: 


= Oo: 


jon 


a 


™ XDM as the login manager, because it’s 
so minimal and easy to lock down. 


®@ IceWM for the windows manager. 
® Nautilus for file management, without the 


desktop feature enabled (no background 
or desktop icons). 


™ A custom, very limited, “Start” menu, 
with only the programs needed available. 


The school year started, and from a 
technical standpoint, the system per- 
formed better than | anticipated. The 
servers were minimally loaded, and | never 
even had to bring the second and third 
LTSP servers on-line. That first day was 
great, but my excitement was short-lived. 

Our teachers hated the thin clients. 
There were nasty complaints, grievances 
and even personal attacks pointed at me. 
In hindsight, | easily could have avoided 
the problems. Currently, four years later, 
almost the entire staff loves the thin 
clients. | have more requests for thin 
clients in the classroom than | have the 
hardware to handle. Now, four years later, 
| see the results | expected that first day of 
school in 2003. 

The following are a few ways to intro- 
duce Linux successfully, specifically thin 
clients, to an organization, while avoiding 
some of the errors we made along the way. 


1. Go with a Grass-Roots 
Campaign 

| started testing LTSP with a willing teacher 
back in 2001 when it was very new. That 
4th-grade classroom found a ton of bugs, 
but worked through them and really loved 
the technology. It was the right idea to 
start in such a way, but my focus was too 
narrow. Multiple pilot programs, with staff 
demonstrations, are a very effective way 
to get people talking positively about 
Linux. With LTSP, pilot programs are easy, 
because a standard workstation class com- 
puter easily can act as a server, supporting 
4-6 thin clients. A few small-scale labs 
really showcase the power of thin clients, 
and they usually are almost free to imple- 
ment. When staff members see firsthand 
the benefits OSS can offer, especially 
when money is tight, they are much more 
willing to try something new. 


2. Don’t Try to Oversell 

Thin Clients, or Linux 

Linux is great. Thin clients are amazing. 
It’s not necessary to claim thin clients are 
better in every way than traditional work- 
stations. | made the mistake of talking 
about all the good things thin-client tech- 
nology meant and failed to point out the 
shortcomings. As Linux users, we're very 
familiar with the reasons to use OSS, 
because we've already learned that the 
benefits outweigh the drawbacks. It’s 


important to let new users judge those 
things for themselves and come to a deci- 
sion on their own (see the sidebar outlin- 
ing some pros and cons of thin clients). 
It's vital to be up front and informative on 
both sides. In fact, stressing the shortcom- 
ings sometimes pleasantly surprises people 
when they realize how unimportant those 
faults really are. 


3. Get Outside Help 

We hired a consultant. In our case, it was 
late in the game, and it was much more 
responsive than proactive. Hiring a profes- 
sional consultant will be money well 
spent. | don’t mean it's necessary to hire a 
consultant due to lack of personal ability, 
but rather as a political move with the 
advantage of another hired brain. The fact 
of the matter is, consultants actually do 
very little of the work. We hired a well- 
known educational technology consulting 
company in Michigan—Trimble 
Consulting. If | could do it over again, | 
happily would have traded one of my 
servers for a single day of their help and 
wisdom. That said, it is important to hire 
wisely. Consultants should be able to do 
the following: 


® Understand Linux, or at least not be 
afraid of it. 


™ Know they're there to help the integra- 
tion process, not determine whether 
it's wise. 


™ Be the bad guys. They should be willing 
to take blame. Consultants are paid a 
lot and are used to taking the heat off 
local technicians. 


® Communicate very well and translate 
technical lingo into lay terms. 


Hiring consultants is different from hiring 
helpers. As | said, on the job work-wise, they 
do very little. The benefits they offer behind 
the scenes are incredible. The credibility they 
add to a project is priceless. 


4. Learn to Explain 
Complicated Technology with 
Simple, Mundane Explanations 
Here are some example explanations: 


® A thin client is basically a remote key- 
board, mouse and monitor that hooks to 
a giant, powerful computer. Several peo- 
ple plug in to the same giant server and 
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H Cost: thin clients are cheap to buy 
and cheap to replace. 


H Consistency: because everyone is 
logging in to the same server 
remotely, the user experience is the 
same, regardless of the location. 


@ Quick repair: if something goes 
wrong with hardware, a quick 
replacement offers the exact same 
user experience, and the tech- 
support people can repair the 
hardware at their leisure. 


HB Quiet, cool and low power 
consumption. 


M@ They are fast! Because thin clients 
perform at the speed of the server, 
the end user gets to experience the 
responsiveness of a $10,000 server 
instead of a low-end workstation. 


H@ Proprietary software is not as widely 
available for Linux. Connecting to 
a Windows Terminal Server helps 
a bit with that, but at a significant 
additional cost. 


@ Floppy, CD and USB Flash drive 
support is fairly good now, but 
the experience is different from 
having a local drive. 


@ Multimedia is the real downside. 
Sound works for some things but 
not for others. Video streaming can 
work for short clips, but not nearly 
as nicely as on a full workstation. 


BH It’s different, and different is 
uncomfortable. 


Even though it sounds backwards, 
my advice is to point out the things 
LTSP can't do. If you educate users 
regarding those faults, all that’s left to 
discover are the wonderful things thin 
clients can do. 
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share the power. 


™@ An Ethernet network is a lot like a garden 
hose. Every thin client’s hose hooks to a 
big valve, so the giant server needs a real- 
ly big hose in order to get the informa- 
tion to all the thin clients. 


™ A USB Flash drive is like a floppy drive 
that holds a lot more information. It’s also 
easier to carry and less likely to lose data. 


You get the idea. Communication is 
key, and being able to communicate very 
complex ideas simply is a huge asset. In 
fact, looking back over the past four years, 
our biggest difficulties were not technical, 
but social. In general, people are not keen 
to change, so helping them understand the 
process makes the change less foreign. It’s 
impossible to over-communicate. 


5. People Like Shiny Things 

Our thin clients were donated Pentium 
133 computers. Even in 2003, Pentium 
133s were ancient. The big problem is 
that they looked as old as they were. In 
fact, most of them had been marked with 
permanent markers, and many of them 
had CD drawers broken off the front. 
They looked terrible. 

So on the first day of school, teachers 
didn't notice the speed that a dual Xeon 
3.2GHz server with 6GB of RAM gives a 
person. The brand-new gigabit backbone 
in the closet didn’t impress them. The 
deafening sounds of cooling fans in the 
server room didn’t make them feel special. 
They saw old, junky computers on the first 
day of school. In fact, the computers 
looked worse than the Macintosh 5400s 
they left behind the year before. | did get 
new 17" monitors, and brand-new optical 
mice (which were pretty new technology 
then), but in the end, they just saw crud- 


dy old desktops. When staff members 
found out the “new” computers couldn't 
even play audio CDs, they were sure 
they'd been sold out. 

In retrospect, for an additional 10% on 
the original purchase, | could have gotten 
everyone an actual thin-client device, such as 
the type sold by the LTSP developers at 
www.disklessworkstations.com. If that 
number would have been 15% more than 
the original, they could have had thin-client 
devices with LCD displays. Never underesti- 
mate the power of fancy new toys to win 
over hearts. 

As | mentioned earlier, our four-year 
transition to thin clients has been a 
bumpy road. In the end, OSS was power- 
ful enough to overcome even my short- 
sightedness. Our staff and our student 
body love Linux. We've expanded our 
number of thin clients from the original 
75 to a couple hundred. This year, we 
introduced our first 24-station thin-client 
lab in the middle school. In fact, this year 
was the first year | needed all three of 
those original servers. I'm ashamed to 
admit that for three years, | had two 
brand-new servers unplugged. Thankfully, 
that's no longer the case. 

—SHAWN POWERS 


Resources 


K12LTSP: www.k12Itsp.org 


LTSP: www.ltsp.org 


Trimble Consulting: 
www.trimbleconsulting.net 


Diskless Workstations: 
www.disklessworkstations.com 


USER FRIENDLY by J.D. “Iiliad™ Frazer 


GOOD DAY EVERYONE. I’M ERWIN SAID THAT HIS NEW AND WHAT BEATIFIC CREATION 
SITTING HERE WAITING FOR PROJECT WOULD BRING DO YOU GIVE UNTO THE 
ERWIN TO FINISH COMPILING WORLD PEACE AND JOY WORLD, ERWIN? 

HIS LATEST COOL LINUX TO ALL. 


\ 
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—ee FINISHED COMPILING. 


They Saicl It 


Support open spectrum. 
Demand real competition. 
Ask for investigations of 

past spectrum auctions and 
the return of frequencies 
obtained through fraud. 
Demand that systems be 
built-out. It will take political 
pressure to create the kind 
of competitive wireless 
market in which something 
like a $30 Internet phone can 
be made available. Time to 
bring it. 

—Dana Blankenhorn, 
blogs.zdnet.com/open-source/ 
?p=1042 


| don't know who discovered 
water, but I’m pretty sure it 
wasn't a fish. 

—Marshall McLuhan, multiple 
places on the Net 


We're continuing to nurse 
along a few basically 15-year- 
old filesystems while we do 
have the brains, manpower 
and processes to implement a 
new, really great one. 
—Andrew Morton, source: 
Jonathan Corbett, presentation 
at CELF, April 2007 


It's so hard to write a graphics 
driver that open-sourcing it 
will not help. 

—Andrew Fear, NVIDIA Software 
Product Manager, source: 
Jonathan Corbett, presentation 
at CELF, April 2007 


Linux cannot give in to binary- 
only drivers. That way leads to 
the end of our free system. 
—Jonathan Corbett, presentation 
at CELF, April 2007 


EmperorLinux 


...where Linux & laptops converge 


Since 1999, EmperorLinux has provided pre-installed Linux 
laptops to universities, corporations, government labs, and 
individual Linux enthusiasts. Our laptops range from full- 
featured ultra-portables to desktop replacements. All 
systems come with one year of Linux technical support by 
phone and e-mail, and full manufacturers’ warranties apply. 


EmperorLinux specializes in the installation of Linux on a 
wide range of the finest laptops made by IBM, Lenovo, Dell, 
Sony, and Panasonic. We customize your choice of Linux 
distribution to your laptop and provide support for: 
ethernet, wireless, X-server, ACPI power management, USB, 
EVDO, PCMCIA, FireWire, CD/DVD/CDRW, sound, and more. 


EmperorLinux offers Linux laptops with unique features. 
Ruggedized Panasonic laptops are designed for harsh 
environments: drops, vibrations, sand, rain, and other 
extremes. ThinkPad tablet PCs are like other laptops, with 
an LCD digitizer for pen-based input both as a mouse and 
with pressure sensitivity for writing and drawing on-screen. 


www.EmperorLinux.com 


Toucan T61/T61ws 


ThinkPad 161/T61ws by Lenovo 

© Up toot5.4" WSXGA+ w/ X@1680x1050 
e NVidia Quadro NVS 140M graphics 
© 1.8-214 GHz Core 2 Duo 
e 512 MB-4 GB RAM 

© 80-160 GB hard drive 

e CDRW/DVD or DVD+RW 
e 5.2—6.0 pounds 

© 10/100/1000 Mbps ethernet 
© 802.11a/b/g (54Mbps) WiFi 
Starts at $1650 


Rhino D830/M90 


Dell Latitude D830/Precision M90 

© Up to 17" WUXGA w/ X@1920x1200 
e NVidia Quadro FX 3500M graphics 
© 1:8-—2.4 GHzCore 2 Duo 
e 512 MB—-4 GB RAM 

© 60-160 GB hard drive 

e¢ CDRW/DVD or DVD+RW 
° 6.3—-8.6 pounds 

© 802.11a/b/g (S4Mbps) WiFi 
e ExpressCard/EVDO 

e Starts at $1445 


Raven X61 Tablet 


ThinkPad X61 Tablet by Lenovo 
e 12.1" SXGA+ w/ X@1400x1050 
¢ 1.6 GHz Core 2 Duo 

e 1-4 GB RAM 

© 80-120 GB hard drive 

¢ 3.8 pounds 

e Pen/stylus input to screen 
e Dynamic screen rotation 

e Handwriting recognition 

¢ X60s laptops available 

e Starts at $2300 


1-888-651-6686 


Model prices, specifications, and availability may vary. All trademarks are the property of their respective owners. 
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REUVEN M. LERNER 


Django Views 
and Templates 


Getting started with Django views and templates, 
with an eye to the Django way of working. 


Last month, we began looking at Django, a popular frame- 
work for creating Web applications in Python. Django has 
the reputation of being the “Ruby on Rails of the Python 
world”, and there is some justification for that description. 
Even though Django was developed in parallel with Rails and 
has many unique features that distinguish it from its Ruby 
counterpart, it’s difficult to avoid comparing the two. 

In the introduction to Django last month, we saw a num- 
ber of subtle, but important, differences between the two 
systems. Django handles a “project” at a time, where each 
project may contain one or more applications. This contrasts 
with the Rails approach, in which there is no real equivalent 
to Django’s projects, because everything is an application. 

We also saw that Django comes with a built-in Web-based 
administrative system. It takes a tiny bit of fiddling with a con- 
figuration file to activate this administrative system, but it pro- 
vides a great number of benefits to any system that uses it. 

Despite their differences, both Django and Rails use an 
approach that is best known as MVC (model, view, con- 
troller) developed in the Smalltalk community but adopted 
by many other languages and frameworks since then. 
These terms, used verbatim in the Rails world, are called 
models, templates and views in the Django world, and 
they form the bulk of a Django-based site. 

This month, we work with templates and views, ignor- 
ing models and the database API until next month. But, 
don't worry; Django’s templates are quite powerful, and 
they provide an interesting lesson in restraint. 


Creating an Application 

Our first step involves creating a new application within 
our project. For a number of reasons, including the sim- 
plicity with which we can create a page, our example 
application is a very simple blog program. We create our 
blog application by switching into our project directory, 
and then by using our management program, manage.py: 


$ cd /opt/atf/mysite 
$ python manage.py startapp blog 


If this executes successfully, Django won't print any 
messages. Rather, we can get a listing of the current direc- 
tory, which should now include a blog subdirectory: 


$ 1s blog 


__init__.py models.py views.py 
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Django is smart enough to stop us from creating an 
application twice, if we try to do so: 


$ python manage.py startapp blog 
Error: [Errno 17] File exists: '/opt/atf/mysite/blog' 


Now, let's create a simple “Hello, world” view, just so we 
can demonstrate that things are working. Thus, we open 
blog/views.py, which starts off empty except for a comment 
indicating that this is the file in which views are defined. 

The simplest possible “Hello, world” view will be 
called index, acting sort of like the index method in Rails 
or the index.html file on a Web site—providing the default 
content when no specific method is invoked for an appli- 
cation. We also have to import the Python module in 
which Django’s HTTP response capabilities are defined. 
When we're done, the entire views.py file looks like this: 


from django.http import HttpResponse 


def index(request): 


return HttpResponse("Hello, world.") 
If our server isn't already running, we can start it with: 
python manage.py runserver 


Or, if we want to run on a publicly accessible IP 
address, rather than 127.0.0.1 (“localhost”), we can do 
something like this: 


python manage.py runserver 69.55.232.87:8000 


URLConf 
To see the output from our view, let's point our browser to 
the following URL, expecting to see “Hello, world”: 
http://69.55.232.87:8000/blog/. Instead, we get an error 
message, indicating that Django has no idea what we're 
talking about. The good news is that this error message, 
which is turned on only when we are developing an appli- 
cation, tells us what we did wrong—namely, that our 
URLConf definition failed to have any entry for blog in it. 
This is a fundamental difference between Django and 
Rails, and it reflects the different philosophies of the two 
communities. In Rails, you expect the system to do the 
right thing by default; you should have to say something 


only when it deviates from that norm. In contrast, Django 
assumes that you want to make everything explicit. 

Among the things that you need to make explicit is 
the way in which URLs are translated into method calls. 
The Django system for doing this is called URLConf, and it 
is a set of regular expressions defined for the entire pro- 
ject. (It functions something like routes in Rails.) If you are 
familiar with regular expressions, it should be pretty easy 
to add or modify URLConf. 

For example, let’s say we want the /blog/ URL to go to 
our blog application. So, we would open the URLConf file, 
which for the mysite project will be in mysite/urls.py. (We 
already modified this last month, when we added adminis- 
trative capabilities to our Django site.) We then add a line 
that looks like the following: 


(r'*blog/$', 'mysite.blog.views. index') 


In other words, if the system sees a URL that begins 
with blog and ends with /, it should invoke the index 
method within views.py in our blog application. And, sure 
enough, as soon as we save urls.py to disk, we can reload 
our URL http://69.55.232.87:8000/blog/, and in our 
browser, we see “Hello, world”. Our Django application is 
starting to come together. 


More Complex URLs 

Things are not that interesting if all we have is a single 
method and if it always does the same thing. For a blog 
application, we'll presumably want to be able to read a 
particular posting, or postings, on a particular day. And 
although we'll get into the model for our blog application 
next month, it stands to reason that this means we'll need 
to be able to request blog postings by an individual ID or 
by date. 

Our current URLConf doesn’t handle such situations. 
Indeed, Django requires that we explicitly indicate each 
possible URL that a user might request and how that URL 
should be handled. So although we have taken care of 
blog/, we need to handle such URLs as: 


“blog/ID 


Luckily, it should be pretty straightforward for us 
to set up a regular expression that captures such func- 
tionality. If we open urls.py once again, we can add 
another statement: 


(r'*blog/(?P<post_id>\d+)/$', 'mysite.blog.views.view_one_posting') 


Here, we see something a bit strange and different, 
namely the use of capturing parentheses, along with 
?P and names inside angle brackets. We tell URLConf 
that whenever we receive a URL that looks like 
/blog/NUMBER, with one or more digits, we should 
invoke view_one_posting. Moreover, because we have 
captured the digits (\d+, in regular expressions) with 


the name post_id, view_one_posting will be invoked 
with an additional parameter of post_id. 

In other words, once we've modified urls.py to include 
the above mapping, we now can go back to views.py and 
say the following: 


def view_one_posting(request, post_id): 


return HttpResponse("You asked for post '%s'" % post_id) 


Then, we can go to http://69.55.232.87:8000/blog/5, 
and in our browser, we get the following response: 


You asked for post '5' 


Notice that in the view_one_posting method, we used 
%s (for strings) to render post_id, rather than %d (for 
integers). This is because parameters and URLs are passed 
as strings, rather than integers or other data types. We 
could, of course, get around this problem by converting 
the string to an integer, but for our purposes right now, 
this is enough. 

Perhaps this goes without saying, but each method in 
our Django application is a Python method whose output 
just happens to be sent to the user's Web browser. You 
can use any number of Python libraries that you want, and 
even access databases, filesystems and remote computers. 
So long as the output is returned as a legitimate HTTP 
response, your method will work. The application we are 
building in these examples represents the tip of the ice- 
berg, as far as implementation complexity is concerned. 


Templates 

If we wanted to, we could create a whole Web site 
using nothing more than the tools we've already seen. 
However, it quickly would become difficult, and even 
tedious, to do so, putting all of our output strings as 
parameters to HttpResponse. The real solution is to put 
the HTML in external files, interpolating variable values 
as necessary. 

There are a lot of different templating solutions out 
there, and each has its advantages and disadvantages. 
The most common type is that used by ASP, PHP, JSP and 
ERb—the last of which is part of Ruby on Rails—in which 
the code is interspersed with the HTML. This type of 
template can be extremely easy for programmers to work 
with, but it can cause trouble when nonprogramming 
designers are involved, or if you just want to have a 
complete separation between code and design. 

Such templates can be used by Django, but they are 
not the default. Rather, Django uses templates that are 
similar in many ways to the Smarty system for PHP, which 
avoids the use of actual code in the template by introduc- 
ing its own, deliberately limited language. So long as a 
view can pass values to a template, there’s no real need 
for a full-blown programming language inside of a tem- 
plate. It’s probably enough to have if/then statements and 
some basic loops. 
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You can use 
any number 
of Python 
libraries that 
you want, 
and even 
access 
databases, 
filesystems 
and remote 
computers. 


This is what Django’s default template system provides. 
In order to use templates, we first modify the settings.py 
file that sits in the project's main directory. We want to set 
the TEMPLATE_DIRS variable, giving it a list of one or more 
directories in which our templates might be found. For 
example, we could set it to: 


TEMPLATE_DIRS = ( 
"/opt/atf/mysite/templates" 
) 


With this in place, now we create the appropriate 
directories: 


$ mkdir -p /opt/atf/mysite/templates/blog 


And, then we create, in that blog subdirectory, a new 
HTML file, which we call view_one_posting.html: 


<html> 
<head> 

<title>One post</title> 
</head> 


<body> 

<p>This is the "view_one_posting" template.</p> 
</body> 
</html> 


Now, we modify views.py, such that our view_one_posting 
method can invoke this template. We add the following 
import statement at the top of the file: 


from django.template import Context, loader 
Then, we modify view_one_posting to be: 


def view_one_posting(request, post_id): 
t = loader.get_template('blog/view_one_posting.html') 
c = Context({}) 


return HttpResponse(t.render(c)) 


Our template is loaded into the variable t, and the 
context—that is, the variable values we want to pass to 
our template—is bound to the variable c. We then tell the 
template to render itself within the context c. In this par- 
ticular example, we aren’t passing any variables in the con- 
text, so it is represented by an empty dictionary. 

And, if we reload the URL /blog/5 on our site, we 
should see the following in the browser window: 


This is the "view_one_posting" template. 


That's certainly better than what we had before, in that 
the contents of the template are easier to handle (by pro- 
grammers and designers alike) than a string inside of a view 
method. But, how do we pass variables to be interpolated? 
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The answer is quite simple. In the view method, we 
can pass any variable we like to the template using the 
context dictionary, in which the keys are the passed 
variable names, and the values are the passed variable 
values. So, we can say: 


def view_one_posting(request, post_id): 
t = loader.get_template('blog/view_one_posting.htm1') 
c = Context({'post_id': post_id}) 


return HttpResponse(t.render(c)) 


If we want to see this value in our template, we can 
view it in double curly braces: 


<html> 
<head> 

<title>One post</title> 
</head> 


<body> 
<p>This is the "view_one_posting" template.</p> 
<p>The post_id is {{post_id}}.</p> 

</body> 

</html> 


And, sure enough, when we render this template, we get: 
This is the "view_one_posting" template. 
The post_id is: ‘5. 


Conclusion 
Django, like Rails and many other Web frameworks, uses 
MVC to divide the work between models, views and tem- 
plates. This month, we saw how to connect a URL to a 
view, how to pass one or more URL parameters to a view 
and how to invoke a template from a view. 

Next month, we will see how to integrate databases 
and data models into a Django application. 


Reuven M. Lerner, a longtime Web/database consultant, is a PhD candidate in 
Learning Sciences at Northwestern University in Evanston, Illinois. He currently 
lives with his wife and three children in Skokie, Illinois. You can read his Weblog 
at altneuland.lerner.co.il. 


Resources 


The main Django site is at www.djangoproject.com. 
The site contains a great deal of documentation, includ- 
ing tutorials and pointers to mailing lists. 


A prerelease copy of the forthcoming Django book 
(to be published by Apress) is at www.djangobook.com/ 
en/beta, and although the book is still unfinished in many 
places, it is well written and includes many examples. 


~~ "One 
PGI Unified Binary” 


Now, PGI’compilers can generate a single PGI Unified Binary executable fully optimized for 
both Intel EM64T and AMD64 processors, delivering all the benefits of a single x64 platform 
while enabling you to leverage the latest innovations from both Intel and AMD. PGI Fortran, 

C, and C++ compilers deliver world-class performance and a uniform development environment 
across Linux and Windows as part of an integrated suite of multi-core capable software devel- 
opment tools. Visit www.pgroup.com to see why the leading independent software vendors in 
structural analysis, computational chemistry, computational fluid dynamics and automotive crash 
testing choose PGI compilers and tools to build and optimize their 64-bit applications. 


— The Portland Group’ 


www.pgroup.com ++ 01 (503) 682-2806 


The Portland Group, Inc. is an STMicroelectronics company. PGI, The Portland Group, PGI Unified Binary are trademarks or registered trademarks of STMicroelectronics. Other brands and names are the property of their respective owners. 
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COOKING WITH LINUX 


MARCEL GAGNE 


One of my 
favorite 
photo 
management 
applications 
and one of 
the finest 
available, is 
digiKam. 
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Figure 1. digiKam is a superb digital photo management suite 
with features you probably hadn't imagined. 


SRS 


It's Like Deja Vu, but 
at a Higher Resolution 


The coolest Linux and open-source applications may be ones you 


already have. 


How can our main server be out of disk space, Francois? 
Last week, we had enough disk left to last us into the next 
decade. What happened? No, mon ami, | am not accusing 
you of anything. I’m just curious if you, perhaps, might 
have an idea. Quoi? You were downloading a few pack- 
ages for the cool apps issue? Let me see. Mon Dieu! There 
are thousands of source packages here! | know you were 
trying to help, Francois, but you didn’t need to download 
every application on SourceForge. | see some ISO images 
here as well. | guess you wanted to find out which was the 
coolest. How many did you get? All of them! Sigh...we’ll 
talk about this later. For now, we need to get ready for our 
guests’ arrival. They will be here any moment. 

Too late, Francois. Look sharp! Welcome everyone, to 
Chez Marcel, home to exquisite Linux and open-source 
software served with the finest wines and enjoyed by the 
greatest of clientele. Your tables are waiting. Please, make 
yourselves comfortable, and my faithful waiter, Francois, 
will fetch the wine. To the cellar, mon ami. | think the 2002 
Vinedos de las Vientos Tannat will surprise many of our 
guests. The Tannat grape is a varietal from Uruguay, and as 
such, it’s unusual for many palates. Drink and enjoy! 

While we await the wine, let’s look at tonight's menu 
of cool applications. Contrary to Francois’ well-meaning 
efforts, you don’t need to spend hours scouring the 
Internet looking for the coolest software. All you need to 
do is spend a little time exploring the applications you 
already have, and you'll discover wonders you didn't know 
existed, right there in front of you. Speaking of my faithful 
waiter, here he is. 


Se Francois, please pour 
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the wine for our 
guests. While you 
enjoy that first sip, 
mes amis, allow me 
to reintroduce you to 
a program you 
already may be using. 
One of my 
favorite photo man- 
agement applications 
and one of the finest 
available, is digikam. 
digikam is a complete 
digital photo man- 
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agement system. Using digikam, you can create collections 
and organize your photos (by date, type and more) in easily 
searchable albums (Figure 1). Selecting and downloading 
images from your camera is easy and fun. You can add 
comments to pictures, adjust light levels, fix red-eye and 
more. It basically has everything you would expect from a 
photo management suite, plus a few additional options 
that make it a contender for the coolness crown. 

As you can see from Figure 1, digikam automatically 
organizes your photos by date, but there are other clever 
ways of organizing them. One way is to create named 
tags. For instance, say you had a couple hundred travel 
pictures, of which 30 or so are from Rome. You could 
assign all vacation pictures a Travel tag and further tag 
those Rome pictures with the word Rome. Next time you 
want to hunt down the shots of your Roman holiday, 
simply click the Rome tag. If one of those images looks 
like something you want to share with a friend, select the 
image, click Image on the menu bar, select Email Images, 
and a dialog appears that lets you add more images to an 
e-mail message or just send out the one you chose. 

There's a problem with having a lot of images though. 
Okay, there may be more than one problem, but if you are 
like me, you have a tendency to copy your photos more 
than once. digikam has a cool feature that lets you locate 
duplicate images. Simply click Tools on the menu bar, and 
select Find Duplicate Images. 

Speaking of multiple images, wouldn't it be cool to 
create an MPEG slideshow—a movie of your photos? Yes, 
indeed, digikKam does that too. Select a group of images, 
click Tools on the menu bar, and select Create MPEG 
Slideshow. A window appears (Figure 2) from which you 
can choose the amount of time a picture is displayed, a 
fade time (if you want one), the video output format and 
more. You also can organize the order in which the pic- 
tures are displayed by moving them up or down in the list. 

You can do many exciting things with digikam, which | 
will let you explore on your own (look under the Image 
and Tools menu for a start), but before we move on, | do 
want to share one more cool feature. It’s not unusual, at 
Christmas time, to share cards with pictures of the family. 
Using digikam, you can go one step further. How about a 
calendar for the coming year with great color pictures 
selected from your collection? 

To create a calendar with digikam, click Tools, then 
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Figure 2. digikam makes it easy to create a movie from your 
digital photos. 


Select Calendar. At this time, the Create Calendar dialog 
appears (Figure 3) from which you can create a personal- 
ized calendar in only a few steps. To begin, choose a paper 
size (US or A4) and the location of the image on the page. 
The latter selection also defines the orientation of the 
page. At this point, you also can choose a font and decide 
whether you want lines surrounding the calendar dates. 
Click Next to continue on to the photo selection screen 
(Figure 4). This is where the real fun takes place. You've got 
12 months and 12 possible images for your calendar. | sug- 
gested family photos earlier, but you could use anything. 
Instead of buying a calendar of artwork or travel photos, 
how about creating your own? Those Roman ruins would 
probably make a nice addition to your office wall. Click on 
each month to add images, or drag them from digikam onto 
the month icons. To clear an icon and select a different pic- 
ture, right-click the month. When you have chosen some- 
thing for all 12 months, click Next. The final screen displays a 
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Figure 3. Create a custom calendar using your own digital photos. 
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Figure 4. Half the fun of creating your own calendar is deciding 
what pictures will represent each month. 


status as each page is generated and an overview of the 
total. When the job is done, click Finish to close the dialog. 
Another favorite application of mine is the amazing 
Amarok, an impressive multimedia player with a host of 
cool features that makes it one of the best multimedia 
players, regardless of operating system. Sadly for a few of 
your friends, you can't get Amarok on the Redmond OS, 
but because you run Linux, that’s not a problem. When 
you are hard at work in front of the computer, it's nice to 


listen to some of your favorite tunes. Depending on where 


you work, you may even have the luxury of singing along 
with the music. But what if you don't know the words? 

Click the Lyrics tab. If this is the first time you've 
looked at this feature, odds are the script manager won't 
yet be running. Click the Run Script Manager button to 
bring up the Script Manager. You don’t need to be listen- 
ing to asong to do this. At any time, you can click Tools 
on the menu bar and select Script Manager from there. 
When the Script Manager window appears (Figure 5), 
you'll see a list of scripts, categorized on the left. Clicking 
the plus sign beside each category opens the list. 


| invite you to highlight a script and click the About but- 


ton to discover what each one does. You also can click the 
Get More Scripts button to discover additional extensions 
and scripts for Amarok. You'll find screensavers based on 
your current music, voting scripts, scripts that print album 


covers and playlists and more. And, of course, there are lyric 


scripts. One already comes with Amarok, but it isn’t neces- 
sarily turned on. Click on Lyrc (under Lyrics), and then click 
the Run button. This activates a script that queries the lyric 
server at lyrc.com.ar. Then, while listening to your music, 
you can click the Lyric tab, and Amarok displays the words 
to the song, turning you into a diva (or divo). 

Sometimes, you need to take your music with you. 
Music devices, such as iPods, are supported using a plugin 
architecture. What are these plugins? The popular iPod is 
supported, as are iRiver, Creative Nomad and several other 
generic audio devices. Depending on your player, you may 
find that it works with a plugin other than the one you 
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Did you 
know that 
K3b, your 

CD/DVD 

burning 

application 
also is a 
great tool for 
ripping audio 
tracks from 
your CDs? 


Some older play- 
ers use memory 
cards to store 
music in the same 
way that digital 
cameras do. You 
still can use these 
with Amarok by 
creating a generic 
device with a 
defined mount- 
point, such as 
/mnt/mp3player. 
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Figure 5. Amarok’s feature set can be extended with scripts. 


expect, so keep trying. For instance, | have an iRiver T10, 
which works using the MTP Media Device plugin. All this is 
to say that if your particular player isn’t listed or doesn't work 
immediately, don’t give up; try one of the other plugins first. 
To use a player with Amarok, click ToolsConfigure 
Amarok. When the configuration window appears (Figure 
6), look at the left-hand sidebar, and select Media Devices. 
If you haven't done so, make sure your music player (iPod, 
iRiver or whatever) is already connected and powered on. 
Depending on the type of MP3 player you are connecting, 
Amarok already may have detected the device, in which 
case you'll see the device identified at the top of the con- 
figuration window with the Plugin already chosen for you. 
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Figure 6. Amarok should be able to detect your music player, 
but if it doesn’t, it’s easy to add it manually. 


lf your player hasn't been detected, you can add a 
device manually by clicking the Add Device button. From 
the pop-up window that appears, select a plugin, provide 
your new device with a name (whatever you like), then 
identify its mountpoint on the system (for example, 
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/mnt/ipod). Click Apply to close the window and return to 
Amarok. Now, to put music on your player, click the Devices 
tab on the far left. You should see your player listed at the 
top of the left-hand pane. Directly above the device name is 
a button labeled Connect. Make sure your device is con- 
nected through its USB cable, and click Connect. 

If you already have tracks loaded on your music player, 
they will appear in Amarok’s left-hand pane, sorted by artist. 
Incidentally, you can use this list to delete songs from the play- 
er by right-clicking on a track and selecting Delete from device. 
On the other hand, if your player is empty, you'll be looking at 
a plain window, waiting to be populated. To add songs, simply 
drag them from your Amarok playlist (the main window to the 
left) into the device window. Once you do, a Transfer Queue 
appears in the lower half of the media browser window on 
the left. At the very bottom of the media browser, you'll see a 
graphical bar showing the amount of storage space on your 
player, the amount available and the space in your transfer 
queue (Figure 7). When you are happy with the playlist in your 
ransfer queue, click the Transfer button at the top of the 
media browser window. A progress bar appears at the lower 
left, updating as each track is copied to your player. 
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Figure 7. Add your tracks, but keep an eye on the available space. 


That's it. You're ready to take your music with you. 
Monsieur Shakespeare said that music was the food of 
love. Now all you need is a little more wine. 

| wish | could spend more time showing you the wild 
and unusual side of what you may have come to think of 
as ordinary, but the hour grows late. Did you know that 
K3b, your CD/DVD burning application also is a great tool 
for ripping audio tracks from your CDs? Simply click Tools 
and select Rip Audio CD. You also can rip a video DVD— 
click Tools and select Rip Video DVD. 

Then there’s Konqueror, the all-around file manager, 
Web browser combination. When using it as a file man- 
ager, you can split the screen left and right by pressing 
Shift-Ctrl-L. Pressing Shift-Ctrl-T splits the screen verti- 
cally. Select a location in each screen (including remote 
FTP or secure copy connections), then simply drag and 


drop from one window to the next. You even can split 
the already-split windows. 

Konsole, your KDE terminal application, can transform 
to a transparent Konsole, allowing your desktop artwork 


Resources 


digikam: digikam.org 

Amarok: amarok.kde.org 

K3b: extragear.kde.org/apps/k3b 
Konqueror: www.konqueror.org 

Konsole: konsole.kde.org 

Marcel’s Web Site: www.marcelgagne.com 


The WFTL-LUG, Marcel’s Online Linux User Group: 
www.marcelgagne.com/wftllugform.html 


to shine through. Click Settings—~Schema, and then select 
Transparent Konsole (or one of the many other looks). 

As you can see, mes amis, although it’s tempting 
(and exciting, as temptation can be) always to be on 
the lookout for the latest, greatest and coolest applica- 
tions, your search need not always be a long one. 
Sometimes the coolest applications really are the ones 
you already use. You just need to spend a little time 
getting to know them better. Uncover their secrets and 
your existing friendship could turn into something 
beautiful. Mon Dieu, | think my mind may be wander- 
ing to places other than software. If so, mes amis, 
surely we can blame the wine. Francois, please, refill 
our guests’ glasses a final time. Raise your glasses, 
mes amis, and let us all drink to one another's health. 
A votre santé! Bon appétit/™ 


Marcel Gagné is an award-winning writer living in Waterloo, Ontario. He is the 
author of the all-new Moving to Free Software, his sixth book from Addison- 
Wesley. He also makes regular television appearances as Call for Help’s Linux 
guy. Marcel is also a pilot, a past Top-40 disc jockey, writes science fiction and 
fantasy, and folds a mean Origami T-Rex. He can be reached via e-mail at 
mggagne@salmar.com. You can discover lots of other things (including great 
wine links) from his Web site at www.marcelgagne.com. 
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WORK THE SHELL 


DAVE TAYLOR 


Well, | can’t 
create a 
casino as 

a shell 
script, and 
| certainly 
can't create 
either a 
secret 
agent or 

a gorgeous 
female 
sidekick, 
but | can 
create a 
Baccarat 
game asa 
shell script. 


Baccarat? 


. 007's Favorite Game: 


Create a shell script to get a taste of being Bond, James Bond. 


| was recently watching Casino Royale and thinking about 
the James Bond series, particularly how Sean Connery was 
so much more sophisticated as Bond than Daniel Craig. 
Connery was more debonair, and one of the ways he'd 
demonstrate it was by playing a mysterious high-stakes 
game called Baccarat while surrounded by gorgeous 
women in casinos in Monte Carlo. 

Well, | can’t create a casino as a shell script, and | 
certainly can't create either a secret agent or a gorgeous 
female sidekick, but | can create a Baccarat game as a 
shell script. Heck, it’s probably the first time anyone's even 
attempted it! 

lf you've been a faithful reader of this column since 
the beginning, you'll know that almost two years ago we 
started out by writing a Blackjack game as a shell script. It 
was a long-winded affair (| didn’t just say that, did |?), but 
as part of the project, we created a simple way to emulate 
a deck of cards, “shuffle” the cards (that is, put them in 
quasi-random order) and even convert a numeric 1-52 
value into a suit and rank. 

We'll use that as the starting point for creating our 
Baccarat game, so we can focus on the complicated rules. 
Let’s start there. 


The Basics of Baccarat 

Baccarat has been around since the mid-1400s, and the 
variation I'll be coding, Punto Banco, is completely rule- 
based with no skill involved. Two cards are dealt to both the 
player and banker and, depending on those cards, a third 
might be dealt to one or both. Face cards are worth zero, 
and numeric cards are worth face value. You add the value 
of a hand and its final value is that value modulo 10. The 
higher point value wins, and if they're identical, it’s a tie. 

For example, if the player was dealt a 7H and a 3C, 
that'd be worth zero (7 + 3 = 10 % 10 = 0). A 6S and 2D 
is better though; it’s worth 8. And, finally, a9+3+J=2. 
Got it? The best possible hand is worth 9 points. 

lf either the banker or player have 8 or 9 points, 
no further cards are dealt to either, and the game ends 
with the dealer or player winning or in a tie. If the 
player has an initial total of O-5 points, the player can 
draw one additional card. 

The banker's play at this point is sufficiently complicated 
that I'll defer explaining it until next issue. For now, let's 
just look at how to code these rudiments of Baccarat. 

Ready, Mr Bond? 
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Capturing the Basics of Baccarat 

The first piece of the puzzle is pretty straightforward—a 
shell function that returns a Baccarat value for a given 
sequence of cards (integer 1-52): 


function handValue 
{ 
handvalue=0 # initialize 
for cardvalue 
do 
if [ $cardvalue -ge @ ] ; then 
rankvalue=$(( $cardvalue % 13 )) 
case $rankvalue in 
0|11)12 ) rankvalue=0 
i ) rankvalue=11 ;; 


esac 


handvalue=$(( $handvalue + $rankvalue )) 
fi 
done 


handvalue=$(( $handvalue % 10 )) 


This function makes it easy to calculate the value of a 
hand—whether it will have two or three cards. Here's a 
typical invocation: 


handValue ${player[1]} ${player[2]} 


The result is returned as the global variable handvalue, 
which is calculated by summing up the individual rankvalue 
of each card. 

Dealing the cards is accomplished by initializing things: 


initializeDeck 
shuffleDeck 


Then, here's actually dealing out the cards from 
newdeck into the player and dealer arrays: 


player [1]=${newdeck[1]} 
player [2]=${newdeck [3] } 


nextplayercard=3 


dealer [1] =${newdeck [2] } 


dealer [2]=${newdeck[4] } 


nextdealercard=3 


Realistically, if it’s a shuffled deck, it would be an identical result to 
have the first two cards go to the player and the next two dealt to the 
dealer, but since we're trying to emulate the actual sequence of events 
at a Baccarat game, I'm dealing cards #1 and #3 to the player and #2 
and #4 to the dealer. 

The next step is to calculate the value of both the player and dealer 
hands, which can be done with the handValue function: 


handValue ${player[1]} ${player[2]} 
playerhandvalue=$handvalue 
handValue ${dealer[1]} ${dealer[2]} 


dealerhandvalue=$handvalue 


Now, let's test to see if we're done with the hand because either 
player or banker has a hand value of 8 or 9: 


if [ $playerhandvalue -ge 8 -o $dealerhandvalue -ge 8 ] ; then 
echo -n "Play is complete. " 

showResult 

exit 0 


fi 
The showResult function simply calculates (and displays) who won: 


function showResult 


{ 


if [ $dealerhandvalue -gt $playerhandvalue ] ; then 
echo "Dealer wins" 
result=1 

elif [ $dealerhandvalue -1t $playerhandvalue ] ; then 
echo "Player wins" 
result=2 

else 
echo "Tie" 
result=3 

ti 


I'll stop here, but next column, I'll pick up the task by examin- 
ing how to test whether the player should get a third card. Then, 
we'll really dig into the rules for the banker and start running 
some games! 

Note: | have leaned heavily on Wikipedia's information on 
Baccarat for the rules and history of the game (en.wikipedia.org/ 
wiki/Baccarat). I’m focused on what's called Punto Banco, the 
so-called North-American-rules Baccarat. If you, like Bond, prefer 
Baccarat Chemin de Fer or Baccarat Banque, you can tweak things 
as necessary. 
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BEACHHEAD 


JON “MADDOG” HALL 


Some of you 
may have 
been a “kid 
at the 
airshow” 
and had 
that feeling 
of cool 

as the 
plane made 
the ground 
shake. 


Cool Change 


Here are some ideas for cool projects; send us some of yours. 


Nick Petreley, our fearless Editor in Chief, had joined me 
at my favorite restaurant and watering hole, Alideia dos 
Piratas for an evening out. As the music played and the 
young folk danced the night away, Nick and | talked about 
the Linux community and all the interesting things that 
have happened there. 

Nick had declared this month as Cool Projects month 
for the magazine. Traditionally, the focus had been Home 
Projects, but he rationalized that although a lot of home 
projects are cool, the name Home Projects was too limit- 
ing. Robotics and wearable computers, for example, are 
cool, but not necessarily home. 

Even before | knew that this month would be Cool 
Projects month, | had started thinking about what makes a 
software project “cool”. 

My “cool” thought processes started with an article by 
Kathy Sierra called “Professional vs Passionate” at the site 
headrush.typepad.com/creating_passionate_users/2006/ 
11/two_simple_word.html. 

Although this particular article focuses on how we 
sometimes discard the raw passion people feel when they 
shout out words not normally found in polite society and 
how companies change when they trade in their sandals 
and T-shirts for the suits and ties of the Professional, the 
article also touched on the concept of what is cool. One 
example given was: “kid-at-airshow-seeing-an-F16-on- 
afterburners-rip-by-so-close-it-makes-your...” 

Some of you may have 
been a “kid at the 
airshow” 
and had 
that feeling 
of cool as 
the plane 
made the 
ground 
shake. 

Now, this 
coincided with 
some thought about 
computer conferences 
and tradeshows and why 
people do or do not go to 
them. Certainly, some people go to 
conferences and tradeshows to find 
existing solutions to existing problems. Other 
people go because they are curious about what is 
going to be a solution in the next one or two years. Still 
others go to find out new and novel things (cool stuff)— 
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things that may never turn into a product or service or 
things that may be suitable only for such a small group of 
people that no one sees a commercial opportunity for 
them. Nevertheless, they're cool. 

Lately, some of the conferences I’ve attended seemed 
to be concentrating on the existing solutions, and less on 
the forward thinking and cool stuff. 

Of course, cool is in the eye of the beholder, and it 
seems to change over time. In my early days of program- 
ming, choosing a new algorithm to solve a problem and 
having the program take half the time to get the same 
answer was cool. Today, this has less of the feeling of the 
F-16, but it is still important. 

So, | went out to my user group and asked, “what's 
cool?” | received several responses before the mailing list 
moved away from the original question onto a discussion 
of one topic that someone thought was cool. 

Some of the ideas were: 


@ NCID: a system for obtaining, displaying and even 
speaking caller-ID information. 


@ MythTV, Plutohome and LinuxMCE: various systems for 
recording, storing and playing back TV programs, 
video, music, pictures and home automation. 


@ Automation of testing procedures to cut production 
testing time by a factor of four and to cut down on 
human error. 


Each one of these projects might be con- 
sidered cool by various people. 
Although automating a testing 
procedure might not sound 
like an F-16 flying at 200 
feet above you, it 
might be a project 
that would get 
some people 
excited and 
try to imple- 
ment that 
solution 
right away. 
Other people 
may not have 
any interest at all 
in these projects. 
Just as | was about to turn away from the responses to 
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my query, someone mentioned the site of C Data 
Solutions, Ltd. (www.cdatas.com/index1.html), where a 
computer was being built out of CompactFlash Modules. 
Now | admit to being a gearhead, particularly when it 
comes to very small systems, but when | saw this, an F-16 
lew right over my head. 

A couple of months ago, | mentioned the Gumstix 
olks (www.gumstix.com) in an article. Both Gumstix and 
he C Data Solutions systems share some of the same 
characteristics—they’re tiny and low power, yet pro- 
grammable through our favorite operating system. | see 
hem, and | get excited over what they could do. However, 
don't remember seeing any projects at any of the recent 
conferences | attended using these types of systems, and | 


problem. It was about watching students work together 
in teams and solve problems as teams. It was an exciting 
time, and it was cool, both for the students and for 
the spectators. 

Realizing that you, my readers, and | may never agree 
on what is cool or why we think it is cool, | will make a 
request. Write to me at cool@linuxjournal.com, and tell 
me about existing FOSS projects that you think are really 
cool (and why), or what would make a really cool project. 
If you have ideas for a really cool contest or conference 
activity, send those too. | will roll them up and post them 
on the Linux Journal Web site. 

But, as you formulate your idea, please make sure to 
duck, so the F-16 will miss your head.m@ 


hink that is a shame. 

At a tradeshow recently, | saw a very nice sponsored 
area where high-school and college students were trying 
0 get their robots to maneuver a maze and do some sim- 
ple tasks. The students were all in teams of three or four 
people, had their own colors, flags and work areas. The 
eams were changing their programs and even doing hard- 
ware changes to their robots right at the event. This event 
was not about selling some product or even solving some 
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such companies as Western Electric Corporation, Aetna Life and Casualty, Bell 
Laboratories, Digital Equipment Corporation, VA Linux Systems and SGI. He is 
now an independent consultant in Free and Open Source Software (FOSS) 
Business and Technical issues. 
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DOC SEARLS 


There was 

a constant 
digital dance 
between 
analysis and 
synthesis— 
and in the 
midst of it 
all, the 
increasingly 
obvious need 
for tools 

that are open 
and free. 


Work to Be Done 


Break down the knowledge barriers with demotic character, relative 
impermanence, dialogic imagination and other picture postcards. 


We were flying from the Coliseum north above Figueroa 
into downtown Los Angeles, about five stories above the 
road surface. After crossing the Harbor Freeway, we passed 
the Bob Hope Patriotic Hall on our right, then barely cleared 
ten lanes of elevated traffic on the Santa Monica Freeway. 
After zooming past the Convention Center and the Staples 
Center, we veered to avoid hitting a large building facing 
south across Olympic that featured monstrous portraits of 
Kobe Bryant and two other Lakers. From here north, the 
street became a slot between rows of buildings on both sides. 
When a break showed up at 5th, we took a hard right to 
head east a couple blocks before taking another right on Hill. 
Below to our right appeared Pershing Square. Suddenly, we 
dove down to the ground and into the escalators leading 
down into the subway. Here we paused on the platform to 
watch a train go by, then punched out through the roof and 
into the air above, where we took Hill North to a right on 4th 
and then across Broadway and through the arched doorway 
into the Bradbury Building, a hollow shrine to ironwork and 
exposed elevator mechanics, best known for the noir scenes it 
provided for the movies Blade Runner and Chinatown. From 
here, we flew out to the north and then east on Cesar 
Chavez to look at plans for turning the concrete trough of 
the Los Angeles River into a green park of some kind. 
Suddenly, the park itself appeared, the river blue to the brim. 
Our pilot and urban simulation god was Zachary Rynew, 

a grad student in Architecture at UCLA. The flight was a 
demonstration of Virtual Los Angeles, a simulacrum of 
downtown LA. One can fly through and explore Virtual LA 
in surreal time—that is, right now or at times in the past or 
future. Virtual LA is designed to facilitate urban planning, 
emergency response, architectural development, education 
and a mess of other specialties and combinations thereof. It 
combines 3-D geometric modeling with street-level and aerial 
photography. Some of the visuals are so detailed, you can 
read graffiti and small signs in shop windows. The system 
originally was developed on SGI systems, but it was moved 
0 Linux several years ago. Like all open-source projects, and 
ike the subject it explores, it is under development. Work 
will continue on it as long as the project is useful. 

Virtual Los Angeles was one among many visualizations at 
he Digital Innovation Day Showcase at UCLA in May 2007. 
Others included the Roman Forum Project, 800 Years of Berlin 
and the Qumran Visualization Project. In UCLA's Visualization 
Portal, we were treated to a tour of Santiago de Compostela, 
a 13th-century cathedral in Spain, as three projectors blended 
images seamlessly on a curved 180° screen surrounding seat- 
ing for 40. In each case, the experts talked about how the 
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work of virtual construction (or reconstruction) resulted in 
better insights and understandings of settings both long gone 
and not yet built. There was a constant digital dance between 
analysis and synthesis—and in the midst of it all, the increas- 
ingly obvious need for tools that are open and free. 

All this work grew alongside and out of a relatively new 
discipline called digital humanities. | had first heard of the 
field from Joseph Vaughan, Assistant Director of the UCLA 
Center for Digital Humanities, in an e-mail responding to a 
Linux for Suits report on the first Desktop Linux Summit, 
back in 2003. In that e-mail, Joe reported on gradual 
progress in getting more Linux and open-source software put 
to use on digital humanities projects. | ran into Joe at the 
showcase, and he said that, in fact, a lot of progress had 
been made during the four years since he sent that e-mail, 
with much more to go. The same could be said of digital 
humanities, and for that matter, of all digital technologies. 

This was one of the first points made by the keynote 
speaker, Dr Willard McCarty of the Centre for Computing 
in the Humanities, Kings College London. Dr McCarty’s 
talk was titled “What's going on?”, and he began by 
telling us that what's happening in computing is by nature 
perpetually protean. “In the history of inventions”, he 
said, “computing is in its infancy, its products incunabular. 
The point | am making is that it and they always will be, 
however progressively better they get.” 

But, he does note some trends: 


About 30 years ago, Northrop Frye noted a striking 
technological change in scholarly resources, from the 
“portly...tomes” of his childhood, communicating in 
their physical stature “immense and definitive 
authority”, to the “paperback revolution” of his 
adulthood, which better matched the speed of 
scholarship and gave wing to the diversification of 
publishing. The demotic character and relative 
impermanence communicated by these paperbacks 
also implied the undermining of authority | just 
mentioned, in this case a weakening of the barrier 
between author and reader. Running in parallel if 
not cognate with this physically mediated change 
came theoretical changes in ideas of textuality, for 
example, Mikhail Bakhtin’s “dialogic imagination”, 
reader-response theory and, more recently, in anthro- 
pological linguistic studies of context. Meanwhile, 
various parts of computer science have developed 
congruently, from design of black-boxed, batch- 
orientated systems of former times to toolkits and 
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implementations of “interaction design”. Computing 
has become literally and figuratively conversational. 


This applies not only to the rise of independent and 
autonomous development by individuals and groups, but 
also to participation by everybody willing and able to 
weigh in, including users, which McCarty places now 
at the start rather than at the end of things: 


.. it makes less and less sense to be thinking in terms 
of “end users” and to be creating knowledge-juke- 
boxes for them. It makes more and more sense to be 
designing for “end makers” and giving them the 
scholarly equivalent of Tinker Toys. But, we must 
beware not to be taking away with one hand what 
we have given with the other. To use Clifford Geertz’ 
vivid phrase, we need rigorous “intellectual weed con- 
trol” against the Taylorian notions that keep users in 
their place—notions of knowledge “delivery”, 
scholarly “impact”, learning “outcomes” and all the 


Resources 


Virtual Los Angeles: 
digitalinnovations.ucla.edu/2007/ccc/projects/Jepson.htm 


Digital Innovation Day Showcase: www.ucla.edu/spotlight/07/ 
digital-innovation.html 


Roman Forum Project: www.digitalinnovations.ucla.edu/2007/ccc/projects/ 
Favro.htm 


800 Years of Berlin: 
www.digitalinnovations.ucla.edu/2007/ccc/projects/Presner.htm 


Qumran Visualization Project: digitalinnovations.ucla.edu/2007/ccc/projects/ 
Schniedewind.htm 


UCLA‘s Visualization Portal: www.ats.ucla.edu/portal/default.htm 


Santiago de Compostela Tour: digitalinnovations.ucla.edu/2007/ccc/projects/ 
Dagenais.htm 


Digital Humanities: en.wikipedia.org/wiki/Digital_Humanities 
Dr Willard McCarty: staff.cch.kcl.ac.uk/% 7ewmcecarty 


Centre for Computing in the Humanities: www.kcl.ac.uk/schools/humanities/ 
cch/index.html 


Kings College London: kcl.ac.uk 
ProjectVRM: projectvrm.org 


Berkman Center for Internet and Society at Harvard University: 
cyber.law.harvard.edu 
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rest of the tiresome cant we are submerged in these 
days. The whole promise of computing for our time— 
here is my historical thesis—is directly contrary to the 
obsolete 19th-century cosmology implicit in such talk. 


McCarty’s proximal context is the academy, particularly 
the graduate schools and programs where specialties tend to 
be well-tended and guarded spaces. About these, he says: 


Particularly since the advent of the Web, our 
attention and energy have been involved with the 
exponential growth of digitization. The benefits for 
scholarship here are unarguably great. But as ever 
larger amounts of searchable and otherwise com- 
putable material become available, we don’t simply 
have more evidence for this or that business as 
usual. We have massively greater ecological diversity 
to take account of, and so can expect inherited 
ways of construing reality and of working, alone 
and with each other, to need basic renovation. 
Here is work to be done. It's not a matter of break- 
ing down disciplinary boundaries—the more we 
concentrate on breaking these down, the more 
they are needed for the breaking down. Rather the 
point is the reconfiguration of disciplinarity. From 
computing’s prospect at least, the feudal metaphor 
of turf and the medieval tree of knowledge in its 
formal garden of learning make no sense. We need 
other metaphors. Here is work to be done. 


The challenge McCarty lays out over and over, with 
those four words—"work to be done”—is not just for aca- 
demics alone, or academics in cahoots with programmers 
and other technologists. It's work to be done by everybody. 

This comes home for me with ProjectVRM, which 
I'm heading up as a fellow with the Berkman Center for 
Internet and Society at Harvard University. Here, I’m work- 
ing to drive development of tools that fix marketplaces by 
giving customers both independence from vendor entrap- 
ments and better ways of engaging with vendors on an 
equal power footing. These tools of independence and 
engagement don’t exist yet—though their parts surely do, 
amid the growing portfolio of free and open-source soft- 
ware and standards that are lying around in the world. 

The problem is, the only code | know is Morse. A few 
years ago, that would have disqualified me as a project leader. 
Now, it doesn’t. Because now, it’s easier than ever for people 
who see problems worth solving to find the people and tools 
that will help those problems get solved. That’s what's hap- 
pening with ProjectVRM. It’s what happened with the user- 
centric identity movement out of which VRM grew as a spe- 
cialty. And, it’s what will lead Linux and other open-source 
projects off of their own turf and out into a larger world 
where users are at the start of things, and not just the end.m™ 


Doc Searls is Senior Editor of Linux Journal. He is also a Visiting Scholar at the 
University of California at Santa Barbara and a Fellow with the Berkman Center 
for Internet and Society at Harvard University. 


Polywell's Ultimate Linux Systems 


More Choices, Excellent Support Service, Great Value! 


1U Value pes Starts at $399, Up to 4GB RAM 


$399 1U-485Ax Sempron 3000+, 1GB DDR2, 80GB HD 
(For Volume Purchase Only) 


$499 1U-485Ax Athlon64 3500+, 2GB DDR2, 80GB HD 
(For Volume Purchase Only) 


$699 1U-690GA Athlon64 X2 Dual-Core 3800+, 4GB DDR2, 
2x80GB HD (Dual LAN +$45) 


$1299 1U-690GA Athlon64 X2 Dual-Core 4200+, 8GB DDR2, 
2x80GB HD (Dual GigaLAN +$45) 


W Elec Servers, Up to 64GB RAM, 41B HD 


$1,999 1U-690S4 Athlon64 X2 Dual-Core 500+, 8GB DDR2 — 
2TB 4x500GB HD, Dual Gigabit LAN 

$2,399 1U-1000SL Opteron 1210 Dual-Core, 8GB DDR2 ECC 

2TB 4x500GB HD, Dual Gigabit LAN 

$2,999 1U-2500A16 2 x Opteron 2212 Dual-Core, 16GB ECC DDR2, 
1TB 4x250GB HD, Dual Gigabit LAN 

$5,999 1U-2500A16 2 x Opteron 2216 Dual-Core, 32GB ECC DDR2, 
2TB 4x500GB HD, Dual Gigabit LAN (Option: 64GB+4TB HD) 


cole Servers, 1U 8-Way Quad Opteron bid? 


$4,999 1U-Twin 2 x 2 Dual-Core Processors, 2 x 8GB ECC DDR2, 
2 x Dual 250GB HD, 2 x Dual Gigabit LAN 


$5,999 1U-8415A 4 x Opteron 8212 Dual-Core, 16GB ECC DDR2, 
1.5TB 3x500GB HD, Dual Gigabit LAN 


$7,999 1U-8415SS 4 x Opteron 8212 Dual-Core, 32GB ECC DDR2, 
2x74GB 15K RPM SAS HD, 3x Gigabit LAN 


(custom config. available) 


1U 8-Way, 5U 16-Way Servers, Up to 128G RAM 


$12,999 1U-8450SS 4 x Opteron 8212 Dual-Core, 64GB ECC DDR2, 
2x74GB 15K RPM SAS HD, 3x Gigabit LAN 


$39,999 1U-8450SS 4 x Opteron 8214 Dual-Core, 128GB ECC DDR2, 
2x74GB 15K RPM SAS HD, 3x Gigabit LAN 


$18,500 5U-8850T5U 8 x Opteron 8212 Dual-Core, 64GB RAM, 
2TB 4x500GB HD, 3 x Gigabit LAN 


$46,999 5U-8850T5U 8 x Opteron 8214 Dual-Core, 128GB RAM, 
4TB 8x500GB HD, 3 x Gigabit LAN 


Blade Servers - 10 Dual or Quad Processors Blades 


$13,999 8U 10 x 2055A Blades 

10 x (Dual Opteron 2210 Dual-Core, 4GB RAM, 80G HD) 
$24,999 8U 10 x 2500M Blades 

10 x (Dual Opteron 2212 Dual-Core, 16GB RAM, 80G HD) 


$36,999 8U 10 x 8450A Blades 
10 x (Quad Opteron 8212 Dual-Core, 16GB RAM, 80G HD) 
$66,999 8U 10 x 8450A Blades 
10 x (Quad Opteron 2214 Dual-Core, 32GB RAM, 80G HD) 


AMD Dual-Core technology enables one platform to 
meet the needs of multi-tasking and multi-threaded 
environments; provides platform longevity 


Polywell OEM Services, Your Virtual Manufacturer 
Prototype Development with Linux/FreeBSD Support 
Small Scale to Mass Production Manufacturing 
Fulfillment, Shipping and RMA Repairs 


Pol 


ywell Computers, Inc 1461 San Mateo Ave. South San Francisco, CA 94080 650.583.7222 Fax: 650.583.1974 


Opteron, Sempron and ATHLON are trademarks of Advanced Micro Devices, Inc.. Quadro, nForce and Nvidia are trademarks of NVIDIA Corporation. All other brands, names are trademarks of their respective companies. 


Linux Appliance starts at $299 


’ Poly 485Ax Sempron 3000+, 512M DDR2, 
ATI X1100 Graphics, 100Mbit LAN, DVD, 
160G HD $299 (For Volume Purchase Only) 


Poly 690GA Athlon64 X2 Dual-Core 3800+ 
1G DDR2 ATI X1250 DVI+VGA, GigaLAN, 
DVD-RW, 500GB HD $499 


- (OEM /ODM Service Available) 


Low-Cost NAS Storage 
2.078 Starts at $999 


$999 Netdisk 4000 2.0TB 4x500G 


$1,550 Netdisk 6000 3TB 6x500G (2U) 


High-Density Multi-Processor Servers 


6TB 2U Storage Server 


2012SC-2055A 


4TB 8x500G, Opteron 
6TB 12x500G, Opteron 


$2,999 
$3,999 


18TB 4U Storage Server 


4024AIS-2500A16 
42TB 24x500G, Opteron $7,500 


48TB 24x750G, Opteron $11,999 


H 20 Years of Customer Satisfaction 
a 5-Year Warranty, Industry's Longest 
© First Class Customer Service 


888.765.9686 


www.Polywell.com/us/LJ —OLy 
POLYWELL 


) NEW PRODUCTS 


Motorola’s RAZR2 V8 


Motorola started tinkering with Linux on mobile phones as an experiment in 
China, which was supported, interestingly, by the Chinese government. Because 
the company was so impressed with Linux, our beloved OS is becoming ever- 
more common on its mobile devices. One of the latest Linux-powered options is 
the RAZR? V8, a next-generation edition of this popular handset line. The RAZR? 
V8 features up to 500 minutes of talk time, quad-band connectivity, up to 2GB 
of memory, a two-megapixel camera, Opera Web browser, independent speech 
recognition and music management based on Windows Media Player 11. 
Motorola was coy about the latter feature, divulging only that it has licensed 
audio codecs, DRM and transfer protocols from Microsoft and integrated them 
into its Linux-Java platform. Still, it calms the Linux-questing soul to know that 
one can intentionally go out and get a butt-kicking Linux-based phone! 


www.hellomoto.com 


Drew Technologies’ 
DashDAQ 


As the Detroit automakers obsessively perfect the cupholder, 
Drew Technologies seeks to sneak a slick Linux-based device, 
the DashDAQ, on board your new ride. The DashDAQ, says 
DrewTech, is “a cross between an automotive gauge, dash- 
board, navigation system, data acquisition system, trip recorder, 
diagnostic tool and a handheld computer.” The device was 
designed for use as an automotive display, includes OBD2 com- 
munications protocols and runs on Linux (yesss!). The software 
that is included with DashDAQ allows users to create their own 
themes and automotive gauge skins. Other features include 
dual-ARM architecture, 4" (QWVGA) 24-bit TFT color display, 

a touchscreen and 64MB of RAM. The product is available 
through resellers, distributors and directly from DrewTech. 


Axigen Mail Server www.dashdag.com 


Axigen continues to add to the feature set of its Mail 
Server messaging solution, with version 4.0 now ship- , 


ping. The key new feature in version 4.0 is a Personal 
Organizer module offering features such as calendar- 


ing, tasks, journal, notes and collaborative support. The oe =. 
feature is available via Axigen‘’s WebMail interface and 

Outlook clients. In addition, the product now contains sah erest 
the Axigen Outlook Connector, which implements 9 en 


most Exchange-like features, such as server-side search 
folders. Axigen Mail Server comes in three additions: 
ISP/HSP, Business and a free Office edition. 


www.axigen.com 
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NEW PRODUCTS |) 


Kyle Wheeler’s Qmail Quickstarter » 
(Packt Publishing) 


If qmail’s home page is too unwieldy for you, pick up Kyle Wheeler’s Qmail Quickstarter, a new 
book for folks familiar with Linux/UNIX and DNS servers who desire to set up a qmail mail serv- 
er. Starting with the basics, Qmail Quickstarter moves on to getting e-mail messages in and out 
of the queue, along with storing, retrieving and authenticating them. The book also covers vir- 
tualisation of domains and user management, filtering spam, SSL encryption and mailing lists. 
Packt says that the book's style focuses on practical examples that system administrators can 
use right away, but that it also explains the rationale behind every example. 


www.packtpub.com 


FORBIODEN LEGO a 
4 Ulrik Pilegaard and Mike Dooley’s 


Forbidden LEGO (No Starch Press) 


Book publishers seem to release their books in waves, and presently we find ourselves in the midst of a 
geek-book tsunami. The ever-eclectic No Starch Press continues to pack its LEGO Mindstorms series with 
fun titles, the latest being Forbidden LEGO by Ulrik Pilegaard and Mike Dooley, subtitled cheekily as “Build 
the Models Your Parents Warned You Against”. Presto, rebellion accomplished! Forbidden LEGO focuses 
on “free-style building” and shows you how to make zany models, such as “a toy gun that shoots LEGO 
plates, a candy catapult, a high-voltage LEGO vehicle, a continuous-fire ping-pong ball launcher and other 
useless but incredibly fun inventions.” A word to the wise: stock up on LEGOs in advance! 


yaw PeRE NTS 


* 
z 


X 


www.nostarch.com 


Data Structures and Algorithms for 
Game Developers by Allen Sherrod P| damsrevcrunss 
' and ALGORITHMS for 
Grand Theft Auto ain't Pong, which is why you'll need this book—Data Structures and Algorithms 
for Game Developers by Allen Sherrod—for your next game undertaking. Published by Charles River 
Media, this title focuses on teaching game developers the fundamentals of data structures and algo- 
rithms using C++ and its alternative options, such as C++ STL. The book also covers many topics 
that today’s game and graphics programmers must know to be successful, including geometry man- 


agement techniques, KD-Trees, binary space partitioning trees, sphere trees and so on. Furthermore, 
the code that’s included is not platform- or OS-dependent, which is just as we Linuxers like it! 


www.charlesriver.com 


ActiveState’s Tcl Dev Kit 


The latest news from ActiveState is the release of Tcl Dev Kit (TDK) 4.0, a multiplatform toolkit for creating and 
deploying Tcl applications. In addition to providing essential tools for building self-contained, installation-free executa- 
bles, TDK includes a graphical debugger, profiling and analysis tools for code optimization, a pre-compiler and a 
Windows services manager. New features in TDK 4.0 include checker enhancements for finding errors in Tcl scripts, 
enhanced TEA package management, compiler support for Tcl 8.5, GUI facelifts and support for Mac OS X and Solaris 
x86. Other supported platforms include Linux, Windows, AIX and HP-UX. 


www.activestate.com 


Dynamic Tools for Dynamic Languages 


ActiveState 


Please send information about releases of Linux-related products to James Gray at newproducts@linuxjournal.com or New Products 


c/o Linux Journal, 1752 NW Market Street, #200, Seattle, WA 98107. Submissions are edited for length and content. 
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SUIILO TOUR CWin 


AND RELIVE 
THE ‘80S! 


Save your quarters; 
you won't need 
them when you 

build your vintage 
arcade player. 


Shawn Powers grow weary of hearing 
how bad Linux is for 
gaming. In this article, 

$ | buck that stereotype 

| 
and show how to use 
penguin power to relive 
the ‘80s. In this cool project, 
| describe how to construct 
a fully functional arcade 
cabinet. When complete, 
you'll be able to play all 
the old coin-op games from 
your childhood in the coin- 
free luxury of your living 
room (or garage—depend- 
ing on the tolerance of 
individual spouses). 

The system uses soft- 
ware based on the MAME 
(Multi-Arcade Machine 
Emulator) Project to play the 
original classics emulated in 
Linux. MAME uses software 
to emulate the original 
arcade hardware. This 
article explains how to 
connect the original joysticks 
and buttons, and even 
gives some pointers 
regarding purchasing rights 
to the original games. 


38 | august 2007 www.linuxjournal.com ILLUSTRATION @ISTOCKPHOTO.COMICYRO PINTOS 


The Biggest Case Mod 

You'll Ever See 

First off, it’s the arcade case that makes this 
project truly awesome. If you don’t have an 
actual arcade cabinet, with original arcade 
controls, you might as well go back to play- 
ing Zork on your TSR-80 for your dose of 
nostalgia. The cabinet is what makes this 
cool, period. It’s up to you to decide whether 
to make a cabinet or buy a used one. If you 
make the whole thing from scratch, the 
design is completely up to you. The down- 
side is that doing this requires some wood- 
working skills. | have no such skills. In fact, to 
me, “saw” is a type of waveform on an 
oscilloscope. So, | opted to purchase an old 
arcade machine and gut it out. Unless you 
are a woodworker, this is the way I'd suggest 
doing it. As a bonus, | got all the buttons 
and controls | needed for free, and the case 
even smelled a little like a pizza parlor. (Okay, 
that might be my imagination.) 

To find used cabinets, look up “coin op” 
or “arcade” in the phone book. It was my 
particular fortune to find a man willing to 
give me two used cabinets for $30. | did 
have to listen to a litany on the downfall of 


Figure 1. You can see the cabinet won't win any beauty contests, but it’s just 
dripping with nostalgia. 


the arcade generation and how kids these 
days don't appreciate the classic games. | 
think he threw in the second case for me 
when | mentioned that | was rebuilding it 
because | wanted to relive my youth. He 
liked that idea almost as much as | did. Keep 
in mind that if you happen to get an arcade 
monitor, it’s possible actually to use it, but it 
requires more work. For the purpose of this 
article, | assume you're using a standard 
computer monitor. 

After | bought my wife flowers and took 
her out to dinner, | was allowed to bring one 
of the cases into the house and start the fun. 
My particular case was from an old Neo-Geo 
game, and | chose to leave the somewhat 
damaged decal on the side (Figure 1). 

If you are, in fact, a woodworker, you 
may want to purchase the particle board and 
make a truly custom cabinet. There are 
designs and original measurement specifica- 
tions for some common arcade machines on 
www.arcadecontrols.com. In fact, that site 
has tons of information on constructing a 
MAME machine from the ground up, and it’s 
an invaluable Web site. To be honest though, 
even if | were able to start from scratch, | 
actually prefer the 
used cabinet. There's 
something special 
about knowing it's 
the real deal, even 
if the insides are 
all different. 

The most difficult 
part of construction, 
or me, was getting 
he monitor properly 
mounted. | used an 
old 17" Apple moni- 
or that was destined 
or the trash. As you 
can see in Figure 2, 
he monitor shelf has 
o be at the right 
angle so that the 
ace of the screen is 
lush with the 
Plexiglas front. I'm 
sure there are wood- 
working shortcuts for 
iguring out how to 
get the angle correct, 
but | ended up with 
some really compli- 
cated geometry 
equations involving 
SINE and COSINE. 
Yep, I'm a geek. 

The shelf below is 


Figure 2. | most likely could have gotten just as 
close with a guess, but | must confess a lot of 
math went into the angle calculation for this shelf! 


Figure 3. | really was lucky with the free buttons 
| got with the cabinets. | couldn't have planned 
the colors any better if | tried. You'll see evidence 
of my woodworking shortcomings by the extra 
hole | drilled. 


to hold the computer. All | did there was line 
up the shelf with the coin door in the front, 
so | could reach in to power the computer on 
and off. (The coin mechanism actually can be 
wired and used, but mine was broken, so | 
never installed it. Maybe next time.) 

The last bit of woodworking is the most 
complicated. You must decide what button 
layout you desire. I’m the kind of person 
who spends 20 minutes looking at a dinner 
menu, so a decision like this was bordering 
on impossible for me to make. | read, | 
Googled, | compared and | surveyed. My 
suggestion is that you avoid all that and go 
with the layout | chose. Why? Because it 
works for most games, and you can always 
change it later. Figure 3 shows the button 
layout | chose, which happens to be a standard 
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FEATURE Build Your Own Arcade Game Player 


OPTIONS 
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Street Fighter II layout. 

You'll notice an extra joystick in the mid- 
dle, which is not part of the Street Fighter 
layout. It’s an original Ms. Pac-Man joystick 
that my wife got as a belated high-school 
graduation gift. (We're an odd family.) It’s 
wired directly to the player-one joystick, so 
don't worry if you don’t have such a beast, 
it’s not really additional, just an alternate. 

The joystick and button holes will vary in 
size, but standard buttons use 1%" holes. 
Feel free to lay the buttons out however 
you like. | went out of my way to make sure 
the buttons were perfectly placed, but there 
aren't rules for these things. Don’t forget a 
button for inserting coins! Mine is sticking 
out of the front of the machine, about 
where you'd put in a quarter if it were an 
actual arcade. 

Now you should have a mostly empty 
case with a monitor shelf, computer shelf 
and a wooden control panel full of holes. If 
you are doing this work in the house, and 
you happen to be married, take this time to 
vacuum. You can thank me later. 

If you are the artistic type and want 
to paint the control panel and cabinet, 
go ahead and do so. | couldn't wait, and 
to this day my cabinet has a raw-wood 
control panel. 


This is where the magic happens. If you're 
anything like me, you've already tried MAME 
on your computer and played the classic 
games with your keyboard. It’s fun, but it's 
definitely not the same as the original. In this 
step, we connect the real arcade controls to 
the computer. First, if you weren't lucky 
enough to get used joysticks and buttons on 
the cabinet, you need to buy some. There are 
a few places to buy arcade controls, but one 
of the most popular is Happ Controls 
(www.happcontrols.com). If you check out 
the controls section of the Arcade Controls 
Web site mentioned previously, you'll find a 
plethora of vendors for such things. 

Once you have the actual buttons and 
joysticks, you need to decide whether you 
want to buy a keyboard encoder or hack a 
keyboard yourself. The former option is 
easier and more reliable, but the latter 
offers a certain geeky charm. | chose the 
latter. There are very in-depth instructions 
on the Arcade Controls Web site regarding 
how to figure out button layouts properly, 
but see the Keyboard Hacking sidebar for 
a quick description, so you can decide for 
yourself which way to go. If you do 


pi ea se A eee SF Pie 
Figure 4. This photo pretty much speaks for itself. 
It's a great example of how not to organize wires. 


Figure 5. My goal was to avoid soldering. The 
spade plugs look great, but they honestly don’t 


make a very good connection—another example 
of what not to do. 


choose to hack a keyboard, | encourage 
you to come up with a better way to 
connect the wires than | did. Figure 4 
shows my horrible rat’s nest of wires. It's 
ugly, but it works. 

Whether you choose to create your 
own keyboard controller or purchase one 
from a place like Hagstrom Electronics, 
you still have to connect the wires to the 
arcade controls. Because I'm terrible at 
woodworking, you'd think there would be 
a certain balance in the universe that 
would suggest | could solder well. Let me 
assure you that balance does not exist. | 
thought it would be really great to avoid 
soldering to all the buttons. | bought 
some spade connectors that would slide 
right onto the terminals (Figure 5). 
Although the idea still sounds great, it 
turns out that the microswitches in the 
buttons and joysticks really need a solid 
connection. Unless you want to take apart 
your control panel to wiggle wires every 
so often (like me), | highly suggest you 
solder everything. Invite a friend, because 
I'm under the opinion that soldering takes 
three hands. Perhaps your two hands are 
better than mine, but at least the friend 
can hold a light. 


Linux Laptops 


Starting at $799 


OBTAINING LEGAL ROMS 


. Buy the actual motherboard from the original game. This may sound far fetched, 

but it’s not too difficult to purchase broken arcade boards from either local coin-op 
vendors or on-line from eBay. Because you're just looking for the legal right to use 
the downloaded ROM, it doesn’t really matter if the board is broken. 


Linux Desktops 


. You can buy the X-Arcade keyboard, which comes with a handful of classic ROMs. Starting at $375 


. Check out arcadecontrols.com. There are some free, noncommercial games avail- 
able for download and some more ideas regarding obtaining legal ROMs. It often 
is difficult to find whom to buy the rights from, even if you want to spend the cash. 


How Many GHz to Emulate dle-of-the-road Pentium 4. If you want my 

2.5MHz? advice, I'd say to use what you've got. If you 

So now, you should have a cabinet, mounted feel the need to invest more later, you always 

controls, some sort of keyboard interface and can do so. 

an empty shelf waiting for a computer. The = 

last step is to put in a computer and install It’s Penguin Time! Li nux Se rvers 
the software. The horsepower you'll need To run MAME in an arcade cabinet, you Starting at $899 

really depends on what games you want to must configure two separate programs: 

play. If you are planning to play the classics, MAME itself and a front end. Because 

like Pac-Man and Asteroids, a Pentium II MAME is a command-line program, you 

should suffice. If, however, you want to play need a front end to launch the games with 

Street Fighter Il, or any newer games, you the joysticks and buttons. This last step is — 
should get a Pentium Ill or 4. | have a mid- really my favorite part. If you thought my 


Vector Games 


DON'T BE SQUARE! 
CET CUBED! 


Asteroids 
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Figure 6. Here's the main WahCade screen. This screenshot was provided by Andy Balcombe, the 
WahCade developer. Thanks Andy! 
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FEATURE Build Your Own Arcade Game Player 


KEYBOARD 
HACKING 


. Take apart the keyboard, making 
sure not to damage the PCB. 


. Find the two sets of connectors 
on the PCB, and on a chart, 
label them X and Y. 


. Trace the very small, fine lines 
on the flexible plastic sheets to 
determine the X and Y coordi- 
nates for every key. No, really, 
I'm not kidding. 


. Solder wires onto each X and Y 
terminal, and use those wires to 
connect to the arcade controls. 


indecisiveness regarding the button layout 
was excessive, you don’t want to know how 
long | spent determining what software to 
run. Obviously, | chose to run Linux. Check 
out the Options sidebar for some compar- 
isons on a few common choices regarding 
distros and front ends. 

| chose to install a stock Ubuntu distribu- 
tion. Really any distro would have worked, 
but | just happen to like Ubuntu’s single CD 
install. Also, the WahCade front end has a 
really nice .deb installer, so Ubuntu’s Debian 
roots really made sense. First, install Ubuntu 
like a normal desktop user. | chose to install 


The system 
uses software 
based on the 
MAME (Multi- 
Arcade Machine 
Emulator) 
Project to play 
the original 
classics emulated 
in Linux. 
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Feisty Fawn, because it’s the most recent 
release, and | happened to have a CD already 
burned. I'd highly suggest running the instal- 
lation using the same monitor you plan to 
use in the arcade cabinet. It will save a lot of 
hassle later on. 

After Ubuntu is installed, open up a 
terminal window, and install xMame (the 
X Window System version of MAME): 


# sudo apt-get install xmame-x 


Then, go to www.anti-particle.com 
and get the latest version of WahCade. You'll 
want to get the .deb file. To install it, type: 


# sudo dpkg -i wahcade-xxx.deb 


Now, you're ready to configure things. 
The installation instructions at the 
WahCade home page are pretty clear, but 
make sure to follow them carefully. You'll 
have to do a few command-line things 
and figure out where you want to store 
ROMs and such. There is a really nice GUI 
configuration tool, in which you can 
assign the proper keys (from your joy- 
sticks) to control the interface. Once you 
have the front end configured and ROMs 
in place, which admittedly will take a 
while, go ahead and fire up WahCade. 
Make sure to add the -fullscreen and 
-skip_disclaimer options for xmame, the 
latter because it's difficult to type OK 
with just a joystick. (MAME displays a 
disclaimer by default that forces you to 
type OK in order to continue.) 

If everything is configured correctly, you 
should see the WahCade program and be 
able to select and launch games accordingly 
(Figure 6). | hate to admit this, but | spent 
hours trying to figure out how to configure 
the buttons for xmame. | tried to edit config 
files, but it was too complicated for me to 
figure out. In the end, it turns out MAME 
has a built-in method for reassigning keys. 
When | realized how simple it really was, | 
could have kicked myself. | still might. Simply 
press the Tab key once you're inside a game, 
and you get a nice configuration screen 
where you can configure either the default 
keys or the game-specific keys. 

One of the beauties of MAME is that it's 
possible to assign a command to a combina- 
tion keystroke. For example, | didn’t want to 
install a button just to exit a game, so | 
mapped “exit game” to pressing Player 1 
and Player 2 simultaneously. It really limits 
the amount of buttons needed. MAME also 
has an option to pause, which is something 


the original arcade never had, but it’s very 
nice for home use. The yellow button on my 
control panel is mapped to pause. 

Once MAME and WahCade are working, 
you need to automate the process. Set the 
GNOME login manager to log in the user 
account automatically, and create an .xsession 
file that avoids the GNOME environment 
and launches WahCade directly. Here’s an 
example of how to create an .xsession file: 


# cat > ~/.xsession 

# wahcade 

# %D (actually press control-D) 

# 

# chmod +x ~/.xsession (this makes 


it executable) 


So there you have it, an arcade system, 
complete with real controls and original 
games. You can customize your system to fit 
your needs. Perhaps you want a headphone 
jack, so you can play without disturbing others. 
Maybe you want to add music support 
and have your arcade machine double as 
a jukebox. Because the cabinet has a full 
computer with Linux under the hood, the 
possibilities are really endless. Have fun! 


Shawn Powers is a geeky Technology Director for a small 
school in northern Michigan. He did manage to find a wife to 
love him and has three wonderful daughters. His wife even 
watches Star Trek with him, but he suspects it’s just because 
she loves him. Send him e-mail at shawn@brainofshawn.com. 


Resources 


MAME Arcade Emulator: 
www.mame.net 


WahCade Front End: 
www.anti-particle.com 


Arcade Controls (Cabinet Building Info): 
www.arcadecontrols.com 


Happ Controls: 
www.happcontrols.com 


Hagstrom Electronics: 
www.hagstromelectronics.com 


Ubuntu Linux: www.ubuntulinux.com 


AdvanceMAME/AdvanceMENU: 
advancemame:sf.net 


LinCade: www.pc2jamma.org 
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CREATE 


A LINUX 
VPN 


for a Nokia E61 
with Openswan 


A VIRTUAL PRIVATE NETWORK (VPN) allows you to send 
traffic across an untrusted network without exposing the content of 
that traffic. Conceptually, this is done by creating a pipe between two 
hosts where all network traffic transferred is protected by cryptography. 

The example in this article is connecting a Nokia E61 device to a 
home network through a VPN over the Internet. The Nokia E61 is a 
smartphone that has Wi-Fi support as well as a VPN client. A similar 
procedure might work for other phone models using the same VPN 
client software, though the hardware was not on hand to test this. 
The Linux side was run on Fedora Linux 6; other distros might have 
slight path and package name changes. 

The VPN support on the Nokia E61 uses IP security (IPSec). 
Openswan is an IPSec server that is configured on the Linux machine 
to provide the other end of the virtual network. 

| should mention one caveat up front: I’ve been unable to 
configure the VPN client on the phone to connect to a server that 
does not have a static IP address. 

To keep notation simple, | refer to the phone as e61 and the server 
running Openswan as vserv. The IP address of the e61 is irrelevant to 
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the article, as you likely will be moving around to different Wi-Fi 
hotspots with the phone. When a VPN is set up, the e61 gets another 
IP address, which the e61 refers to as the virtual IP address. Once the 
VPN is set up, this virtual IP address is where all traffic to and from the 
e61 is sent. For this article, | use a 192.168.x.x IP address for this e61 
VPN address. As the non-VPN IP address of the e61 is mostly irrelevant, 
unless | explicitly mention otherwise, the e61 IP address will be this 
non-Internet-routable IP address. 

Unlike the other network settings on the phone, you cannot 
configure the VPN manually using the e61 itself. You have to create a 
package containing all the information about the VPN and install that 
package on the phone. These packages are the SIS files. A VPN SIS file 
also must be digitally signed before the e61 will allow you to install it. 
Signed SIS files normally have an sisx extension. The most difficult part 
of setting up the e61 to talk to Openswan is in creating the sisx file to 
install on the phone. 

The SIS file still must be digitally signed, even if you have set 
the configuration parameter Software installation to All in App 
Mgr/Options/Settings. 


The Contents of the 
sisx Package 
The sisx package is composed of three files. 
Two of these are boilerplate-type package 
metadata (the VPN.pin and VPN.pkg files). 
Getting the boilerplate files out of the 
way, the VPN.pin file is mostly uninteresting 
and is shown in Listing 1, and the VPN.pkg 
file is shown in Listing 2. Both files should 
work fine without any changes. Note that 
the paths shown in Listing 2 are to be inter- 
preted relative to the phone itself and should 
not be changed. 


Listing 1. Some Very Basic Package Metadata 


POLICYNAME] 

VPN public 
POLICYDESCRIPTION] 
VPN public 
POLICYVERSION] 

ileal 

ISSUERNAME] 

Do not edit 
CONTACTINFO] 

Do not edit 


The VPN.pol file shown in Listing 3 
defines the meat of how to connect to the 
VPN and what key to use for authentication. 

Some things need to be changed in 
VPN.pol before using it. The main changes 
are the static IP address of the Openswan 
server (192.168.0.1) and the password to 
use to connect. The server's IP address 
appears more than once in the configura- 
tion file. To avoid any confusion about 
virtual IP addresses mentioned above, this 
IP address is the one from which vserv can 
be reached publicly from the Internet. The 


Symbian Developer Certificate Request - InstallShield Wizard 


Figure 1. Installing the DevCertRequest Application in Wine 


password is in the last field: the 
KEY. The number is the string 
length of the key that follows 
after a space. 

If USE_XAUTH is set to true, 
when establishing the VPN 
connection the e61 prompts you &EN 
for a user name and password with 
which to connect. This provides 
an additional level of security. In 
the event that the e61 is stolen, 
the thief will have to know your 
user name and password in order 
to access your VPN. 

Openswan can use either 
PAM or a separate config file to 
test the user name and password 
on the server (more on this later). 


; LANGUAGES 
; - None (English only by default) 


Listing 2. Package manifest and description about the appli- 
cation type. You can change the “VPN Policy” string that is 
right before 0x1000597E to something else. 


:"VPN public" 


%{"VPN public"} 


; A VPN POLICY PACKAGE 


; INSTALLATION HEADER 


Creating the sisx VPN 
Configuration File 
Three Windows executables are 
used to create a signed SIS file. 
DevCertRequest.exe is used 
once to create a certificate to 
sign the SIS file; makesis.exe 
and signsis.exe then are used to 
create the package and sign it. 
The last two commands are part 
of the S60 SDK available for free 
from Nokia’s Web site. All of 
these Windows executables can 
be run in Wine, though you need 
to have MFC42.DLL and 
MSVCP60.dll available to run 
DevCertRequest. 

It's best to get the certificate 
in order to begin; register for free 
at symbiansigned.com, and 
download the DevCertRequest 


; - Only one component name is needed 

to support English only 
7 = UID asi the UIDof the 

VPN Policy Installer application 
#{"VPN Policy"}, (0x1000597E),1,0,1,TYPE = 


a LUST Or IPulbles 


5 Folicey wile 
"VPN.pol"-"C:\System\Data\Security\Instal1\VPN. pol" 


; Policy-information file 
; - NOTE: The policy-information file 
MUST be the Last file in this Vist 


7 EM ACRTEEMIME) passes: thet tile: tol the 
respective MIME handler 
; (in this case, the VPN Policy Installer 
; application). 
"VPN.pin"-"C:\System\Data\Security\Install\VPN.pin", 


executable. Registration requires FM, “application/x-ipsec-policy-info" 
your name, 
e-mail 
address, ; REQUIRED FILES 
organiza- - The VPN Policy Installer application 
tion, (0x1000597E), 1, 0, 0, {"VPN Policy Installer"} 
address and 
phone [0x101F7961], 0, 0, 0, {"S6@ProductID"} 
number. 
DevCertRequest have to install this application and then 


is used only to input 
a few settings and 
generate a key and 
a certificate sign 
request (.csr file). 
Unfortunately, the 
DevCertRequest 
executable is actually 
an installer, so you 


execute it (Figure 1). For this article, 
DevCertRequest_30_10_2006_v2.0.exe 
was used. 

After all the pain of installing 
DevCertRequest, using it consists of five simple 
steps, and it isn't needed again afterward. 
You give the location for the new csr file 
(monkeyig.csr); the location for your new 
private key (monkeyiq-private-key.key) and 
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FEATURE Create a Linux VPN 


Listing 3. Policy for the VPN: Where to Connect, How to 


Do It and How to Tell It’s Me 


SECURITY_FILE_VERSION: 3 
[INFO] 

VPN 

[POLICY] 

Seni pse Gale —ant 

esp 

encrypt_alg 12 
max_encrypt_bits 256 
auth_alg 3 
identity_remote 0.0.0.0/0 
Site Sacer jie 
hard_lifetime_bytes 0 
hard_lifetime_addtime 3600 
hard_lifetime_usetime 3600 
soft_lifetime_bytes 0 
soft_lifetime_addtime 3600 
soft_lifetime_usetime 3600 
} 


Feito C-C.O,0) O00) = iC ljasere Cy 26), yp 


inbound = { } 
outbound = { } 


[IKE] 


ADD Riewel O26 SOM leon 2b o2 oro 


MODE: Main 
SEND_NOTIFICATION: TRUE 
EDINA S ili 

FQDN: MobileGroup 
GROUP_DESCRIPTION_II: MODP_1536 
USE_COMMIT: FALSE 
IPSEC_EXPIRE: FALSE 
SEND_CERT: FALSE 
INITIAL_CONTACT: FALSE 
RESPONDER_LIFETIME: TRUE 
REPLAY_STATUS: TRUE 
USE_INTERNAL_ADDR: FALSE 
USE_NAT_PROBE: FALSE 
ESP_UDP_PORT: 0 
NAT_KEEPALIVE: 60 
USE_XAUTH: TRUE 
USE_MODE_CFG: TRUE 
REKEYING_ THRESHOLD: 90 
PROPOSALS: 1 

ENC_ALG: AES256-CBC 
AUTH_METHOD: PRE-SHARED 
HASH_ALG: SHA1 
GROUP_DESCRIPTION: MODP_1536 
GROUP_TYPE: DEFAULT 
ELEERIMESKBYMES 90 
LIFETIME SECONDS: 28800 
PRF: NONE 
PRESHARED_KEYS: 

FORMAT: STRING_FORMAT 
KEYe2 3.100 


46 | august 2007 www.linuxjournal.com 


a password for it; your 


country, state and compa- Listing 4. Extracting the makesis and signsis Executables 
ny; the IMEI of your 

phone (as DevCertRequest mkdir /tmp/e61-sdk 

tells you, keying *#06# on cd /tmp/e61-sdk 

the phone will show it) unzip /tmp/S30_3rded_f0___S6@-SDK-0548-3.0.zip 
and which capabilities you unshield x data2.cab 

want for your certificate; cd Epoc32/tools 


and a confirmation that 


mkdir -p ~/e6ltools 


the information is correct. cp makesis.exe signsis.exe ~/e61tools 


To create the certificate 
itself, you have to return to 


symbiansigned.com and Listing 5. Rolling the Signed Package File 


upload the csr file. First log 


in, and then select the My wine makesis.exe VPN.pkg VPN.sis 
Symbian Signed tab. In the wine signsis VPN.sis VPN.sisx monkeyig.cer \ 


side panel, the Developer 
Certificates option has the 
Request sub-option. At the 
bottom of this page, you can 
upload the csr file (Figure 2). The 
next page allows you to download 
your certificate (Figure 3). 

The makesis.exe and 
signsis.exe files can be extracted 
from the “S60 Platform for 
Symbian OS” SDK, as shown 
in Listing 4. 

With the certificate 
(monkeyiq.csr) file, you now 
can roll the sisx file with the code 
shown in Listing 5. Make sure 
he three files that make up the 
package use the carriage return 
plus new line combination to 
erminate each line instead of the 
standard Linux new line only; see 
unix2dos(1). These three files are 
he pol, pkg and pin files shown 
in Listings 3, 2 and 1, respectively. 


Installing the VPN sisx 
and Final e61 Setup 
Any method can be used to trans- 
er the sisx file to the e61. I've used 
Bluetooth push, in which case it 
can be installed on the e61 directly 
rom the incoming messages list. 
As this sisx file contains a pass- 
word, it is better to transfer it to 
he phone using a wired method. 
Using a mini-SD card in the 
e61 and plugging in the USB con- 
nection cable to the phone and a 
Linux machine likely will bring up 
a file browser for the mounted SD 
card on the e61. Copy the file to 
a convenient location, such as 
Documents/vpn on the e61, and 
eject or unmount the SD card to 


monkeyiq-private-key.key my-certificate-password 


Figure 3. Download the Certificate Needed to 
Sign SIS Files 


force a disk sync before removing the cable 
(Figure 4). 

Once the sisx file is copied to the e61 
memory card, the Menu/Office/File Manager 
on the e61 lets you navigate to the VPN direc- 
tory on your memory card. When you click the 
joystick on the VPN sisx file, the phone asks if 
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Figure 4. Copy the VPN file to a subdirectory of Documents on the memory card. 


Figure 5. Starting the VPN Policy sisx Install 


you want to install it. Right after clicking on 
the sisx file, you should see something like 
that shown in Figure 5. After inspecting some 
metadata, you'll see the ominous-looking 
screen shown in Figure 6. As you have just 
created the package from a bunch of text files 
and you've looked over them for nasties, this 
security warning shouldn't really be much of a 
problem to ignore. 

The VPN sisx file can be prepared for 
use by going to Menu-+Tools—>Settings= 
Connection—VPN. Select VPN access points 
and Options—>New access point. Set the con- 
nection name to something memorable, and 
set the policy name and access point. A 
convenient access point is EasyWLAN. You 
also might want to set the proxy server address 
and port. It's nice to be able to surf the 
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Figure 6. As we made the package, we trust that 
this is not really a security issue. 


Internet and get to Intranet servers over the 

VPN. Directing all Web traffic to the VPN has 
the added bonus that the Wi-Fi hotspot you 

are using isn't able to record the Web sites 


you visit. The final stage is shown in Figure 7. 


If you are already using WEP to connect 
locally and want to continue to do so and be 
able to test the VPN locally, define another 
VPN access point, setting its Internet access 
point to your WEP access point. Having the 
second VPN config means you won't be 
prompted for the WEP key when connecting 
locally. There is little gain in doing VPN over 
WEP except for not having to loosen the 
security on your wireless access point. 


Packages are shown at rpmseek.com 
for Fedora, Mandriva and SUSE Linux. 
Debian.org also lists an Openswan package. 
On a Fedora Linux machine, Openswan can 


Directing all Web 
traffic to the VPN 
has the added 
bonus that the 


Wi-Fi hotspot you 
are using isn’t able 
to record the Web 
sites you visit. 


Figure 7. The final setup—give a name and Wi-Fi 
access point to the policy, and maybe set a proxy 
server too. 


be installed simply by using yum install 
openswan. As mentioned previously, | used a 
Fedora Linux machine for this article; other 
distributions may have subtle differences. 

The two main areas for configuring 
Openswan are the /etc/ipsec.conf file and a 
handful of files in /etc/ipsec.d. The main config 
ile can be left as it stands. A few settings that 
might be of interest are forwardcontrol=yes 
o turn packet forwarding on and off when 
Openswan is started and stopped. The other 
interesting option is the interfaces setting, 
allowing you to control which IPSec interface 
is bound to which network interface—for 
example, interfaces="%defaultroute 
ipsec2=ethl ipsec3=ppp1". If no inter- 
aces parameter is specified, Openswan 
works on the network interface that has the 
default route. For Internet VPN connections, 
his is fine. 

Another parameter that might come 
in handy in the ipsec.conf file is setting 
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FEATURE Create a Linux VPN 


Listing 6. The Openswan Configuration File 
to Allow the e61 to Connect 


conn e61 
# Key exchange 
ike=aes256-shal-modp1536 
# Data exchange 
esp=aes256-shal 
# Authentication method PSK 
authby=secret 
auto=add 
keyingtries=3 
# Modeconfig setting 
modecfgpull=yes 
pfs=no 
rekey=no 
eftid=@monkeyig.example.org 
eft=%defaultroute 
eftsubnet=192.168.0.1/0 
eftrsasigkey=none 


eftmodecfgserver=yes 


eftxauthserver=yes 
rightrsasigkey=none 
right=%any 
rightxauthclient=yes 
rightmodecfgclient=yes 
rightsourceip=192.168.6.252 
rightsubnet=192.168.6.252/32 


Listing 7. Private Key for the VPN 


= IPS Tee” 


plutodebug=all, and reading your syslog 
files if you can’t connect. 

To describe a connection to Openswan 
for the e61, create a config file /etc/ipsec.d/ 
e61.conf, as shown in Listing 6. The pfs 
setting is for perfect forward security. 
Unfortunately, I've had no luck using this 
option and connecting from the e61. As 
shown in the VPN config for the e61, I’ve 
listed the left value as %defaultroute, so 
Openswan will substitute the IP address of 
the network interface to which the default 
route points. As the default route is to the 
Internet, this works well. I've also used the 
DNS name of the vserv as leftid; this should 
be optional. You need to substitute your DNS 
name for monkeyiq.example.org in the 
config file. The rightsourceip is the virtual IP 
address that the e61 will use when talking 
over the VPN. For the firewall rules (Shown 
later), | have assigned the hostname for the 
e61 to 192.168.6.252 in /etc/hosts. 

The same private key that was specified in 
the KEY field of the VPN policy above should 
be placed into the /etc/ipsec.d/e61.secrets file, 
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shown in Listing 7. 

Finally, the USE_XAUTH 
option in the VPN policy needs 
Openswan to have a user name 
and password lookup for this 
connection. The Openswan 
README.XAUTH file recom- 
mends against using PAM for 
this. The password file can be 
created using htpasswd from the 
Apache package, as shown in 
Listing 8. A sample of the passwd 
file is shown in Listing 9. 


A Hole in the Wall— 
iptables 

Trying to debug packet logs on 
the machine that is running as 
the IPSec server can be a little 
difficult. Packets that arrive 
encrypted are decrypted and put 
onto the network interface to 
appear as though they have 
arrived without any encryption. 
Part of a packet log is shown in 
Listing 10, showing a packet 
that appears to have come from 
the e61's IP address without any 
encryption. The log is further 
complicated because the WEP 
setup gives the e61 the same IP 
address as the VPN does. For 
traffic from the Internet, the top 
two packets will have a random 
IP address instead of the e61. To 
get a clearer picture of the packet 
data, connecting the e61 
through a laptop to vserv allows 
proper packet snooping on the 
laptop. The imaps packet in 
Listing 10 will not be seen by 
the laptop—only a bidirectional 
stream of ESP packets. 

The iptables commands shown 
in Listing 11 provide a base to 
allow the e61 to connect to the 
IPSec server from the Internet. 

The packet filtering rules are 
quite simple. Allow Internet 
Security Association and Key 
Management Protocol (ISAKMP) 
packets to enter and exit the 
server and allow any 
Encapsulating Security Payload 
(ESP) traffic, assuming that the 
IPSec server will be responsible 
for sorting out fraudulent packets. 
VPN traffic itself is sent through 
the ESP packets; the ISAKMP is 
used at the VPN session startup, 


Listing 8. Making the initial passwd file with htpasswd. The -c 
option creates the passwd file if it doesn’t exist or replaces it 
if it does. Use the -c only once. Make sure your umask is set 
first, as some distributions have lax defaults. 


umask 0027 


htpasswd -b -c passwd ben your -password-here 


htpasswd -b passwd chidori her-password-here 


Listing 9. The XAUTH user name and password lookup file—the 
password itself is linuxjournal. 


ben: n@taOujOp2G4g:e61 


Listing 10. Partial Traffic Log for ethO on the IPSec Server 


USE 5807 UP ipseesenv = eel: VESPSpil—sr,))eslenhe 
T5e6 iO/7ene LPe ols ipSecsenv; sESP(Spil— er) eas Lennar 
USE SS Ose Re olinav 3.9 a PSCCSCRVEnIMADS amr maCkewn: 


Listing 11. Punching a little hole in the firewall. Note that the 
e61 is set in /etc/hosts to 192.168.6.252. 


ip es -X REMOTEVPN_INPUT 2>/dev/null 
ip es -X REMOTEVPN_OUTPUT 2>/dev/null 
ip es -N  REMOTEVPN_INPUT 
ip es -N  REMOTEVPN_OUTPUT 


ip -I INPUT -j REMOTEVPN_INPUT 
ip -I OUTPUT -j REMOTEVPN_OUTPUT 


ip es -A REMOTEVPN_INPUT -p esp -j ACCEPT 
ip es -A REMOTEVPN_INPUT -m udp -p udp \ 
--dport isakmp -j LOG 
--log-prefix "incoming-ipsec-key " 
ables -A REMOTEVPN_INPUT --src e61 
-p tcp --dport imaps -j LOG \ 


--log-prefix 
ables -A REMOTEVPN_INPUT -m udp -p 
--dport isakmp -j ACCEPT 
ables -A REMOTEVPN_INPUT --src e61 
--dport imaps -j ACCEP 
ables -A REMOTEVPN_INPUT --src e61 
--dport smtp -j ACCEPT 
ables -A REMOTEVPN_INPUT --src e61 
--dport squid -j ACCEP 
es -A REMOTEVPN_INPUT --src e61 


incoming-imaps 


j LOG --log-prefix "e61-strange " 


es -A REMOTEVPN_OUTPUT -p esp -j ACCEPT 
es -A REMOTEVPN_OUTPUT -m udp -p udp \ 
--sport isakmp -j LOG \ 


--log-prefix 
ables -A REMOTEVPN_OUTPUT -m udp -p udp \ 
--sport isakmp -j ACCEPT 


outgoing-ipsec-key 


Listing 12. Remove the e61 access. 


iptables -D INPUT -j REMOTEVPN_INPUT 
iptables -D OUTPUT -j REMOTEVPN_OUTPUT 


and those packets also are logged, so that 
the syslog monitor can alert you of strange 
connection attempts or door knocking. 

Because the non-encrypted traffic is 
placed on the network interface when 
Openswan is done with it, the rules have to 
allow e-mail and squid connections from the 
e61 IP address. | need to have these rules 
here because normally no connections can 
be initiated on the network interface con- 
nected to the Internet. If you filter outward 
traffic too, you have to allow packets from 
these services to the e61 to be sent to the 
Internet network interface (to be encrypted 
by Openswan before being sent to the proper 
Internet address). 

The firewall rules are designed to be used 
with a default policy of drop. The logging 
commands can be added or removed to help 
debugging by searching for the relevant log 
prefix in /var/log/messages to see which 
packets are moving around before the fire- 
wall may drop them. The script shown in 
Listing 12 undoes what was done by Listing 
11 to disable remote access again. 

You also might want to consider using a 
Single Packet Authorization (SPA) client on 
the e61 and setting up the server to open the 
firewall ISAKMP port only after a successful 
SPA. See Michael Rash’s “Single Packet 
Authorization” article in the April 2007 issue 
of Linux Journal for more information on SPA. 


Connecting! 
One more complication exists for using some 
of the publicly available Wi-Fi hotspots. 
Depending on where on the globe you are, 
many of these hotspots follow the pattern 
that when you try to open a Web site, they 
redirect you to their Wi-Fi login page, you 
authenticate to them, and then you can use 
the Internet. If you simply open up a VPN 
access point on the e61 that is set to use the 
EasyWLAN as its Internet access point, things 
will not work. The e61 will start the Wi-Fi 
connection and immediately try to send data 
to set up a VPN connection. As you have to 
authenticate with the Wi-Fi hotspot before 
this, it will let traffic through, but then things 
will come crashing down. 
A way to get around this is to open the 
Web browser and directly connect just using 
EasyWLAN without any VPN whatsoever. Once 
you have authenticated to the hotspot, leave 
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Figure 8. Once the VPN is set up, XAUTH user 
name and password verification starts. 


Figure 9. Success! We can read Linux Journal 
through the VPN link. 


the browser running and use the menu key to 
get back to the main menu, and then open 
the e-mail client. For the access point this time, 
use the VPN that has EasyWLAN set as its 
Internet access point. The existing Wi-Fi con- 
nection is reused, and the VPN is layered on 
top. To get secure Web browsing, you can 
then leave the e-mail program by holding the 
menu key and going back to the browser. Exit 
the browser, and the still-running e-mail pro- 
gram holds the VPN open. Start the browser 
again, and select the VPN as your access point. 
Of course, if the Wi-Fi network you are 


connecting to allows connections without this 
preamble, opening any application that wants a 
data connection should allow you to select the 
new VPN as your access point. Also, if the Wi-Fi 
hotspot remembers your MAC address and 
allows reconnection without explicitly having to 
log in each time, you can start the VPN directly 
on subsequent connections. 

Once the VPN has connected to vserv, the 
e61 prompts you for the user name and pass- 
word to use for XAUTH verification (Figure 8). 

After XAUTH verification, you should be 
able to use the VPN without noticing it. In 
this case, | can browse the Internet using my 
LANs proxy server to fetch the data (Figure 9). 


What’s Next 

Being able to use a DNS name in the e61 VPN 
policy would be wonderful for folks who don’t 
have cheap access to static IP addresses. I'm still 
investigating how to connect using public key 
cryptography instead of the preshared key as 
shown in this article. For connecting a single 
e61 to the network, using a large enough 
preshared key should still be quite secure. 

The information in the article comes 
with no guarantee of being correct, secure 
or suitable for anything; use it at your own 
risk and discretion.@ 


Ben Martin has heen working on filesystems for more than ten 
years. He is currently working toward a PhD combining 
Semantic Filesystems with Formal Concept Analysis to 
improve human-filesystem interaction. 


Resources 


Symbian Signed: symbiansigned.com 


Symbian SDK on Unix HOWTO: 
www.koeniglich.de/sdk2unix/ 
symbian_sdk_on_unix.html 


Symbian SDKs: 
www.forum.nokia.com/main/ 
resources/tools_and_sdks/index.html 


Nokia VPN Client How-To: pipip.de/ 
index.php ?section=other&sub=nokia 
_vpn_en 


Wine and MFC42.DLL: 
bugs.winehq.org/ 
show_bug.cgi?id=4461 


“Single Packet Authorization” by Michael 
Rash (Linux Journal, April 2007): 
www.linuxjournal.com/article/9565 
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Expensive Choice 


Finally, a company that understands Internet media distribution. JAMES LEES 


On April 2, 2007, Apple CEO Steve Jobs along with EMI CEO Eric Nicoli 
announced DRM-free downloads at a new cost of $1.29 US—a fee that's 
$.30 more than the current DRM-formatted music. Shouldn't music be DRM- 
free anyway and not come at an extra cost? That's not what iTunes is saying 
or showing, as it still is going to sell DRM music for $.99 and DRM-free 
music for even more. Why put up with these over-priced songs, not only from 
iTunes but also from other on-line stores, such as Microsoft's MSN Music that 
sells over-priced music, and like iTunes, the music comes with DRM. 

The new DRM-free songs on iTunes will be in MP3 and WMA formats 
and supposedly will be of “superior quality”. That mention of superior quality, 
however, is merely to cover up the extra price and save iTunes from the UK's 
fair trading pressure of infringement on European trading regulations. 
Why should you have to deal with a company that makes you pay more 
for freedom? Freedom is not something you should have to pay for. Why 
pay for freedom when you can choose an on-line site that gives you options 
the other record-label companies don’t offer? 

Such a site exists, and it gives you many choices, including how you 
want to receive an album, such as a downloaded DRM-free format (MP3, 
WAV, OGG, FLAC or AAC) or a purchased CD that will be mailed to you, all 
without the extra “freedom fee”. This site also lets you choose to pay what 
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you think a particular album is worth, and it gives you the option of re- 
downloading a purchased song if you lose it due to a faulty hard drive or 
misplaced file. This site also gives you the option to back up music on another 
device, without restrictions on the number of devices on which you can store 
your music—something iTunes does not allow, and it’s something | know | 
want to be able to do with music | purchase. And, the best thing of all, this 
site allows you the freedom to share an album and give the gift of music to 
your friends. If you buy an album, you can share it with up to three friends 
(based on the honor system). 

What if | tell you it’s all possible with Magnatune—the company 
that says “We Are Not Evil”, and it’s just that, not evil. Magnatune 
allows customers to do everything mentioned above—plus more. It's 
not only an independent, on-line record label, but also a company that 
wants to change how music is distributed and open up possibilities for 
less-known bands and artists. The artists and bands on Magnatune are 
not highly rated with albums or songs in the Top 100, but they still are 
great to listen to and download. All Magnatune needs is only one top 
artist to switch over to put it on the map as a mainstream label company. 
Magnatune takes big strides to label great-sounding artists, and one out 
of 300 artists gets signed. 


What Magnatune Believes 
This is straight from Magnatune'’s site: 


@ All music should be shareware. Just as 
with software, you want to preview, 
evaluate and pass along good music to 
others—in the process of buying it. 


@ Find a way of getting music from musi- 
cians to audiences that's inexpensive and 
supports musicians. Otherwise, musical 
diversity will continue to suffer greatly 
under the current system where only 
mega-hits make money. 


™@ Musicians need to be in control and enjoy 
the process of having their music 
released. The systematic destruction of 
musicians’ lives is unacceptable: musicians 
are very close to staging a revolution (and 
some already have). 


—@ Creativity needs to be encouraged; today’s 
copyright system of “all rights reserved” is 
too strict. We support the Creative 
Commons “some rights reserved” system, 
which allows derivative works, sampling 
and no-cost noncommercial use. 


Magnatune in the Open 
Source Community and 
Becoming a Freedom Fighter 
for Music and Media 

Magnatune recently has taken part in the 
Linux community, allowing you to search and 
listen to its podcast and music directories 
without purchase from the KDE Amarok 
1.4.4 player. This try-before-you-buy philoso- 
phy gives you the opportunity to listen to not 
just one song from an album, but the full 
album, before deciding how much you are 
willing to pay for it—if you choose to pur- 
chase it at all. The great thing with 
Magnatune’s search-and-buy feature being 
put into the Amarok player is that it allows 
you to purchase an album from the player 
instead of going to the site, making it 
easy to search Magnatune’s massive album 


“Listen to more than 500 hand- 
picked complete albums. If you like 
what you hear, download an album for 
as little as $5 (you choose the price), or 


buy a real CD, or license our music for 
commercial use. You'll get MP3s and 
WAVs, and no copy protection (DRM), 
ever.” (From magnatune.com.) 


collection for specific genres, such as 
Classical, Electronica, Jazz and Blues, Metal 
and Punk, New Age and Rock. Once your 
purchase is complete, the song or album can 
be put into any type of music player, such as 
an iPod or any other MP3 player. As | said 
before, Magnatune lets you pay what you 
think the album is worth. It does have a 
starting price of $5, with a maximum of $18 
(the typical purchasing price for an album is 
$8). Each song is DRM-Free (at the price of 
about $.50 a song) and can be downloaded 
in many formats to work on just about any 
music player and device. The best part is that 
50% of the sale price of each album goes 
directly to the artist. 

Another great thing about Magnatune 
that blows iTunes out of the water is its daily 
free song give-away, compared to iTunes 
weekly song give-away. Some people hear the 
word free and think something must be 
wrong or of poor quality, but people who use 
or develop open-source and free software 
know that's not the case. What makes 
Magnatune's daily DRM-free music give-away 
special is not only freedom, but also that the 
songs are great to listen to and have amazing 
sound quality. But, that’s not all, Magnatune 
changes genres every day for the give-away. 
| plan to visit the site on a daily basis to 
download the free song and browse for 
other songs. 

Magnatune also has joined and supports 
the partly open-source Second Life, which 
has become an on-line phenomenon that 
allows you to live a second on-line virtual 
life, while making money on-line and joining 
groups and meeting new people. If you're a 
member of the Second Life world, 
Magnatune allows you to choose a genre 
to listen to while you're in the game or 
in Magnatune's furnished tents, giving you 
a relaxing area to hang out with other 
Magnatune users, making it a better 
community yet again. When you teleport to 
Magnatune’s hang-out location, don’t forget 
to get a free Magnatune T-shirt for your 
avatar. Best of all, Magnatune considers 
Second Life’s playing of its streams to be 
promotional use of its music and grants 
you a license to use that music for free. 
Magnatune also says it will grant you a 
collecting society waiver, so that if ASCAP 
or BMI ever asks you to pay for the music 
you're using on Second Life, you can point 
them to magnatune.com/info/second _life. 

Open Music has become one of the 
largest projects that’s made Magnatune so 
well known in the music industry and Open 
Source community. Open Music is music that 


Figure 1. The Amarok Media Player Playing a 
Magnatune-Provided Preview 


Figure 2. “We are not evil.” 


is shareable, available in “source code” form, 
allows derivative works and is free of cost for 
noncommercial use. It is the concept of 
open-source computer software applied to 
music. Think of Open Music as open-source 
software, allowing you not only to edit and 
sell it, but also to share the wealth with the 
artist and Magnatune, or just give it back to 
the community so others can then edit what 
you edited to make it even better. And, it’s 
all under the Creative Commons, including 
Magnatune’s main site. One example of this 
Open Music Project is the new Amarok Live 
CD Project. Magnatune allowed the Amarok 
Project members and community to bundle 
tracks from the Magnatune store that were 
licensed under the attribution/noncommercial/ 
share-alike version of the well-known 
Creative Commons license. 


The Business Model behind 
Magnatune 

Besides joining the Second Life community and 
starting the Open Music Project, Magnatune 
now has a MySpace site with more than 1,300 
friends. Another on-line community site 
Magnatune uses for advertising and marketing 
is YouTube. Yes, Magnatune has a YouTube 
page that has, at the time of this writing, 17 
videos that include information on Magnatune. 
And, you can watch and listen to artists from 
Magnatune on YouTube, over and over again, 
without having to purchase or pay a royalty to 
watch the video. Because those free services 
are used for marketing, the company and 
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“This is very exciting news about 
Magnatune. This is precisely the kind of 
innovation that will solve the current cri- 


sis in music.” —Lawrence Lessig, founder 
and chairman of Creative Commons 


artists save money that they would normally 
need for getting their names and labels out to 
the public. You also can be a part of the 
Magnatune's marketing team by requesting 
free recruiting cards or printing Magnatune 
mini-posters and handing them out. 
Magnatune’s founder John Buckman’s 
idea was to target people who listen to 
music in the background while they do other 
work (while writing this article, I’m listening 
o Rocket City Riot, Last Of The Pleasure 
Seekers from Magnatune) or music that gets 
little radio airplay or major record distribu- 
ion, but that has a fairly large audience. 
Targeting those audiences is what keeps 
Magnatune’s business going in the long run. 
One of the biggest ways Magnatune stays 
in business is the commercial licensing it offers 
or the music it sells. It allows the music to be 


@Magnatune 


Figure 3. Magnatune’s Web Site 


used in films and television, on the Internet and 
for presentations. And, the license contracts are 
royalty-free, meaning you don’t have to pay 
more if a project is successful. The license and 
the cost to use the music depend on the pro- 
ject. Magnatune gives you a price quote in a 
matter of seconds from the site, and the prices 
are a lot better than other commercial licensing 


fees and are 30% lower than industry standard. 


Music is not the only way Magnatune 
makes money. It sells other merchandise, such 
as posters, clothing and mugs. As with the 
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100MBPS DEDICATED CISCO PORT 
1300GB THROUGHPUT INCLUDED 


albums, 50% of those sales go to the artist. 


Running on Open Source 
Magnatune does not just support open 
source, it also runs on it. Its servers run Linux 
with the most widely available HTTP server 
on the Internet—Apache 2 supporting and 
using PHP and OpenSSL. Magnatune uses a 
MySQL database to store and log purchase 
information, and it's also used for searches 
on the main site, such as for an artist or 
track. When the search is done, a Perl script 
creates the track listings and playlists so 
transactions can be made. 


Conclusion 

So, if you're an open-source advocate and like 
to listen to music that’s not mainstream while 
supporting artists instead of putting money in 
the label companies’ pockets, visit 
Magnatune’s Web site and sign up for the free 
song of the day and maybe buy some albums 
while you're there. And, if you’re an artist, 
Magnatune is just the place to check out, 
especially if want to get your feet wet in the 
music industry. Magnatune is there to help 
artists reach audiences and receive a profit 
(note that albums must have at least 40 min- 
utes of music). Then, you can listen to your 
album on Amarok’s open-source media player 
while making an average of $1,500 US 
(according to Magnatune) if your album is 
accepted. So let's forget about iTunes and 
make the migration to a community-based 
project with a great business model. 


James Lees currently is a Web site developer, and he’s about 

to go to college in Network Engineering. He's also a computer 
hobbyist and die-hard Linux user who wants to help promote 

freeware and open-source software. And, let's not forget, he’s 
also a longtime Linux Journal reader. 


Resources 


Magnatune’s Main Site: 
www.magnatune.com 


agnatune in Second Life: 
Wwww.magnatune.com/info/second_life 


Magnatune’s Open Music Section: 
www.magnatune.com/info/openmusic 


agnatune’s MySpace Site: 
myspace.com/magnatune 


Creative Commons: 
creativecommons.org 


Second Life's Main Site: secondlife.com 
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Do more. 


Kai Staats discusses the 
power and challenges of a 
PlayStation 3 supercomputing 
cluster. JAMES GRAY 


n October 2006, Terra Soft announced 
its plan to build the world's first super- 
computing cluster using the Sony 
PlayStation 3 (PS3), which utilizes the 
IBM Cell Broadband Engine and the Linux 
operating system. The idea emerged when 
Sony Computer Entertainment came 
knocking on Terra Soft's door, interested in 
showing that the PS3 is more than merely 
a game box. After building a 3,000-sq-ft 
supercomputing facility, located at Terra 
Soft’s headquarters, and adding a heavy 
dose of good old-fashioned tinkering, the 
cluster is well underway. Terra Soft’s CEO 
Kai Staats called the building of the PS3 
cluster a “highlight of [his] time in this 
industry”. We caught up with Kai recently 
for an insider’s view on the PS3 cluster. 


LJ: Thank you for agreeing to talk with 
us, Kai. Tell us, why did Sony come to 
Terra Soft to build this cluster? 

KS: Terra Soft has, for eight years, dedicated 
itself to the Power architecture, providing a 
leading Linux OS for systems built upon the 
IBM and Freescale CPUs, such as Apple’s 
PowerPC product line. This experience and 
expertise gave Sony the confidence that Terra 
Soft would provide a high-quality end-user 
experience with professional support. 
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LJ: The PS3 cluster you have created 
together with Sony is an interesting 
application of what is marketed primarily 
as a home-entertainment machine. 
The PS3 is really a flexible, powerful 
machine, isn’t it? 

KS: Yes, the PS3 is both. | believe we are 
experiencing an interesting paradigm shift, 
from three decades of personal computers 
competing with dedicated game boxes to 
the industry's first game box offering true 
personal computer functionality. 

Sony recognizes that, with its Cell 
Processor, the PS3 is not just another image 
processing engine, but a full-featured, fully 
capable home computer and lightweight 
development workstation. This is a tremen- 
dous market differentiator. 

At home, the PS3 elegantly consoli- 
dates the CD, DVD, MP3 player and home 
computer into a single “appliance”. In 
supercomputing, the PS3 offers an inex- 
pensive, lightweight compute node. Not 
designed to compete with the Mercury 
and IBM Cell blades, the PS3 enables 
individuals and labs to develop and optimize 
code for this new nine-core architecture 
within a limited budget. The same code 
seamlessly migrates to the high-performance 
Cell products. 


LJ: We’re curious to know more about 
the significance of the Cell Processor. 
KS: The PS3 is built upon the Cell 
Broadband Engine, a nine-core CPU 
designed by Sony, Toshiba and IBM (the 
STI consortium). It provides an exceptional 
front-side bus performance. 


LJ: Does the Cell’s 1+8 multicore pro- 
cessing environment behave like a 
true eight-core processor, or is there 
a significant difference? 

KS: The first core is the PPU, an IBM 
970-compatible unit. This means any Linux 
application designed to function for the 
Apple G5 or IBM JS21 (for example) will 
operate seamlessly on this core. The addi- 
tional eight SPEs (Synergetic Processing 
Engines) provide eight additional cores that 
may be addressed as CPUs (as compared 

to DSPs), enabling a unique and powerful 
single-chip processing environment. By 
keeping the code on the front side (as com- 
pared to dropping down to the north bridge 
as with historic, multiple CPU configurations), 
the performance is maximized. 


LJ: A critical part of realizing the PS3 
cluster appears to be Y-HPC, your 
cluster-construction suite. What are 


. 


ron ~ 


Terra Soft’s innovations there? 

KS: Simply stated, the Y-HPC cluster- 
construction suite delivers node images to 
compute nodes. But, around this core function 
is the means to manage multiple, unique 
node images and node “personalities” that 
modify any given node image to perform 
various designated tasks. Nodes may be 
deployed as an NFS server, storage server or 
compute node (for example), based upon the 
personality configuration. 

Y-HPC integrates a full command-line 
syntax as well as a graphical user inter- 
face. And, Y-HPC may be deployed, both 
server and nodes, on x86, x86-64 and 
Power (G3, G4, G5, IBM JS20/21, p5 and 
Cell). Furthermore, Y-HPC is the first and 
only cluster-construction suite for IBM, 
Mercury and Sony Cell. 

Although Y-HPC does incorporate some 
basic cluster node monitoring tools, it is 
not designed to replace Cluster Resources 
Moab. Instead, it is designed to integrate 
with Torque and Moab for a complete 
build-to-run solution. 

Currently in a beta v2.0 release, Y-HPC 
is being shipped to key customers in addi- 
tion to its deployment on our internal 
PS3 cluster. 


bike 


PP TTT reser... 


LJ: How does Yellow Dog Linux play into 
other Cell-based systems? 

KS: In the fall of 2005, Mercury Computer 
engaged Terra Soft to develop and maintain a 
commercial Linux OS for its Cell-based systems. 
This was first announced at SC2005, Seattle 
(www.terrasoftsolutions.com/news/2005 
/2005-11-15.shtml). 

Mercury began shipping IBM BladeCenter 
form-factor Cell blades in January 2006 with 
Yellow Dog Linux pre-installed. Terra Soft con- 
tinues to maintain and develop Yellow Dog 
Linux for Mercury's Cell-based products, with 
forthcoming support for the PCI form-factor 
CAB and 1U-rackmount form-factor “pizza 
box" node (mc.com/products/boards.cfm). 


LJ: You’ll be using this cluster for 
bioinformatics. Can you explain for 
our readers what bioinformatics is? 
Why does this particular cluster lend 
itself to this application? 

KS: Wikipedia explains, “The terms bioinfor- 
matics and computational biology are often 
used interchangeably. However, bioinformat- 
ics more properly refers to the creation and 
advancement of algorithms, computational 
and statistical techniques, and theory to solve 
formal and practical problems posed by or 
inspired from the management and analysis 


of biological data.” 

| would add, at a more basic level, 
bioinformatics includes the comparison of 
gene sequences between two or more 
organisms. For instance, as a bacteria or 
virus mutates, one or more of its genes 
differs from the previous strand. Once 
sequenced (the process by which the 
genes are identified, labeled and placed 
into a database), bioinformatics offers a 
means by which the pre-mutation and 
post-mutation gene sequences can be 
compared and better understood. 

Y-Bio and the cluster offer a means by 
which thousands of gene sequences may 
be compared on a daily basis, in addition 
to other applications that will be intro- 
duced by the Department of Energy and 
“dot-edu” researchers. 


LJ: Originally, Sony contracted with you 
to build two clusters based on the PS3 
platform: a test cluster playfully dubbed 
E.coli and a production cluster called 
Amoeba. Is this what finally transpired? 
KS: This original plan was to build both a 
test and production cluster from “beta” PS3 
units, the 2U-rackmounts that Sony provided 
to the game developers prior to shipping 
hardware. Last fall, Sony determined it 
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would prefer to use shipping PlayStation 3 
units, the same as those found in retail 
stores worldwide. 

This renewed effort was put in motion 
in January with the first 20 of a slated 128 
nodes having arrived a little more than two 
weeks ago. 

The first 20 nodes are on the rack shelves 
now. Two weekends ago, we created a 
reduced-footprint Yellow Dog Linux node 
image of roughly 680MB, just the essentials 
for a functional, flexible compute node. 
We are now updating our Y-HPC cluster- 
construction suite, with the folks from 
Cluster Resources applying Torque Resource 
Manager and the Moab Cluster Suite. 


LJ: What has your experience been with 
the cluster thus far? 

KS: The only true challenge was in work- 
ing with a new bootloader (kboot) and the 
associated ramdisk image for netbooting. 
An afternoon or two of tweaking and 
breaking things before we found the 
magic combination of settings, and the 
PS3s were up and running as NFS-booting 
cluster nodes. 


LJ: Can you give us an idea of what 
kind of performance improvements 
you achieve with the nine-core Cell 
Processor vs. other CPUs? 

KS: During our Hack-a-Thon, we witnessed 
some interesting advances in code perfor- 
mance on the Cell Processor. In particular, 
the Mesa library experienced an 80x 
increase over the published performance on 
an Intel Woodcrest. Some folks from IBM 
have been working on BLAST for Cell with a 
noted 10-20x performance improvement. 


LJ: Did you have to do anything special 
to the Linux kernel to exploit the Cell 
Processor’s multicore architecture and 
support chips? 

KS: We do not modify special kernel inter- 
faces for accessing the Cell’s SPEs, which 
have been included in the Linux kernel for 
some time now (| can’t remember when Cell 
support was first added). We also include 
the Cell SDK that allows you to build, exe- 
cute, run and debug applications that utilize 
the Cell’s SPEs. [This question was answered 
with support from Owen Stampflee, Lead 
Developer of Yellow Dog Linux.] 

PS3 has just under 256MB of user-space 
RAM, where the IBM and Mercury Cell 
blades currently offer 512MB per CPU for a 
combined, shared 1GB of RAM on a dual- 
Cell (18-core) board, and far less RAM than 
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what we have come to expect of modern 
32-bit desktops and 64-bit workstations. 

This limited RAM is constraining and, of 
course, things are not quite as snappy as 
with more. But, the 3.2GHz CPU and fast 
front-side bus do compensate well, as the 
desktop is quite usable. Even large footprint 
apps, such as OpenOffice.org, are functional. 
MythTV is impressive. But, with a very large 
image, The GIMP certainly would take a hit. 

When optimizing code for the Cell 
SPEs, independent of the PS3 or IBM 
implementation, it is imperative that the 
algorithms themselves be reworked to 
enable pipelining, the continuous, steady- 
streaming of code and data through the 
relatively small cache. A stall in this 
pipeline, and performance is lost. 

This may be an afternoon of rework, a 
few weeks or more, depending on the com- 
plexity of the code. But when successful, the 
end result is phenomenal with 32-bit floats 
taking advantage of both the AltiVec unit 
onboard each SPE and then the eight-way 
multicore spread. 


LJ: It is impressive to see that you are 
co-developing and open-sourcing a 
range of life-sciences applications in 


tandem with universities and national 
labs, such as Lawrence Berkeley, Los 
Alamos and Oak Ridge. What progress 
have you made here? 

KS: Everything we house in our on-campus, 
3,000-sq-ft server room is given to 

the members of our HPC Consortium 
(www.hpc-consortium.net) free of 
charge. This currently includes seven IBM 
Cell blades (used for Cell code develop- 
ment and optimization) and the growing 
PS3 cluster interconnected via GbE to 

two G5s, a devel box and head node. 
Access is granted via a dedicated port 

on our fibre drop, which currently sits 

at 10Mb and scales in minutes to as high 
as 100Mb, if needed. 

We are expecting to receive additional 
Cell pSeries, perhaps some GPU systems in 
the not-so-distant future. 

All Consortium technical members (those 
whose proposals for Cell development have 
been accepted) are granted an account on all 
in-house systems. 

The Consortium is an experiment in 
leveling the playing field, bringing together 
developers from all programming back- 
grounds and engaging them on the same 
hardware, in the same lists—advanced pro- 
grammers from IBM working with newbies 
from universities, DOE lab and commercial 
employees collaborating during the Hack- 
a-Thon and then assisting each other with 
ongoing development. 


LJ: Are there other applications 
besides bioinformatics that you will 
be targeting? 

KS: Absolutely. At the Hack-a-Thon, there 
were projects to optimize the kernel, to 
build a Cell development toolset for 
Windows, the optimization of Mesa (men- 
ioned earlier), visualization libraries and 
more. There are new projects in motion 
o bring multimedia apps to Cell, CFD 
ibraries and film rendering. The potential 
is limited only by the determination of the 
individuals and teams that work with Cell, 
and the Consortium by no means limits 
he effort to a single field of study. 


LJ: Thank you for sharing information 
about this innovative project, Kai. Good 
luck to you! 

KS: Thank you, too! 


James Gray is Linux Journal Products Editor and a graduate 
student in environmental science and management at Michigan 
State University. A Linux enthusiast since Slack 1.0 in 1993, he 
currently lives in Lansing, Michigan, with his wife and kitty. 
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INDEPTH 


Building a Next-Generation 
Residential Gateway 


You may not need as much as you think to build a residential gateway. ALEXANDER SIROTKIN 


Before embedded Linux became the de 
facto standard for networking devices, building 
a residential gateway (RG) or similar appliance 
used to be an expensive and nontrivial task. 
Real-time operating systems (RTOSes) used on 
these kinds of devices, such as VxWorks or 
pSOS, are relatively expensive and lack many 
features needed for an RG, which has to be 
purchased separately. VxWorks 5.5, for exam- 
ple, comes with a TCP/IP stack that has perfor- 
mance problems and does not even implement 
an L2 bridge, not to mention a firewall. This 
situation created a business opportunity for a 
number of companies, such as Ashley Laurent 
and Jungo, who sell RG software stacks for 
various OSes. The advent of embedded Linux 
distributions, such as uClinux, reduced the 
process of building an RG to choosing the 
right hardware, writing the drivers for periph- 
erals and adding some kind of Web-based 
configuration utility. Embedded Linux reduced 
the development and cost of RG devices not 
only by providing many important features 
that either were missing or expensive in 
RTOSes, but also by easing the integration 
and debugging of drivers and applications, 
thanks to the clear POSIX driver model and 
kernel/user-mode separation. 

However, Linux dominance in the embed- 
ded networking market is now challenged by 
new technologies. In the past, RG devices 
with ADSL, DOCSIS and 802.11 peripherals 
rarely were required to pass throughput 
beyond 10Mbps. New technologies, such as 
802.11n and PON, make 100Mbps through- 
put a reality, challenging embedded engi- 
neers with the task of creating RGs capable 
of handling much higher traffic on the same 
or at least similarly priced hardware. 


Anatomy of the Residential 
Gateway 
Before designing your own residential gate- 
way, it may be worth taking a closer look at 
one of these devices. 

If you decide to take a look inside your 
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Ethernet 
Switch 


Figure 1. RG Hardware Block Diagram 


DOCSIS/xDSL 


802.11 access point or ADSL gateway, which 
probably will void your warranty, you will find an 
embedded board with a CPU (probably some 
MIPS variant), Ethernet switch, ADSL/DOCSIS 
and 802.11 chips, as illustrated in Figure 1. 
Some (or even all) of the above can be 
integrated into one piece of silicon to form a 
System-on-Chip (SoC). On the other hand, you 
also may find the same stock mini-PCI (mPCl) 
802.11 card connected to an embedded board 
as the one you will find in your laptop. 
Chances are, your home gateway runs 
Linux. If you'd like to know for sure or 
maybe hack a bit on it, you have to solder 
the UART connector, which is relatively easy 
(most boards come with UART enabled, but 
without a connector). The software stack 
usually includes some version of Linux (2.4.x 
kernels are still pretty common in the embed- 
ded world), the usual user-space utilities and 
libraries, peripheral drivers and a configura- 
tion utility—either command-line interface 
(CLI), Web or both, as shown in Figure 2. 


Software 

Now that you have a general picture of what 
makes a residential gateway, and assuming 
you have working hardware (more details on 
hardware later in this article), let’s see what 
pieces of software you'll need. At the bare 
minimum, you need kernel sources, some 


BusyBox 


Linux Kernel 


Device Drivers 


Figure 2. RG Software Block Diagram 


variant of the libc library, basic user-mode 
utilities (at least a shell) and a cross-compiler 
toolchain to build them all. In short, you 
need an embedded Linux distribution. 
uClinux is a good choice, as it supports a 
wide range of CPUs (not necessarily CPUs 
without MMU, as the name might imply), 
comes with many useful user-mode utilities 
and has an easy-to-use kernel-style configu- 
ration system for all the packages. As always, 
the choice is not limited only to one distribu- 
tion. There are a variety of free and commer- 
cial Linux distributions from which to choose. 

Assuming your board (not just the CPU) 
is supported by uClinux, the process of 
building a working image boils down to 
downloading the distribution itself and the 
cross-compiler toolchain for your CPU and 
then simply following the build instructions. 
If your board is not supported by uClinux, 


chances are the board manufacturer will 
provide a board support package (BSP)—that 
is, adapt uClinux (or some other Linux version) 
o that hardware. If this is not the case, 
you have to write the BSP yourself, which 
is beyond the scope of this article. 
The major components of the uClinux 
distribution are, of course, the kernel itself, 
he uClibc library and BusyBox. The uClinux 
kernel has support for MMU-less CPUs, but 
his feature is of minor importance nowa- 
days. Because the cost of adding the MMU is 
so small, | expect that most, if not all, 
embedded CPUs used on an RG will have it. 
Nevertheless, uClinux is a great distribution, 
even if you don’t exploit the MMU-less CPU 
support. The same goes for uClibc. It origi- 
nally was created to support MMU-less sys- 
tems, which, for instance, cannot have a 
fork(2) system call. But, even if you don’t 
need this functionality, it is a great alternative 
for glibc on embedded systems, as glibc has 
much larger RAM requirements. BusyBox is a 
collection of standard UNIX utilities, opti- 
mized for embedded systems with low RAM. 
It comes with uClinux, and as with uClibc, 
you usually will prefer it over full-featured 
standard utilities, unless you have a system 
with enough RAM (typically above 16MB). 
You need two important pieces of soft- 
ware that are not a part of uClinux. The first 
piece is a bootloader, which is software that 
usually resides in ROM (at least partially) and 
is responsible for loading the Linux image 
rom Flash to RAM and performing some 
hardware initializations. Unfortunately, there 
is no standard bootloader for uClinux. In 
act, there are no standards for bootloaders 
in the embedded world, and you cannot use 
PC bootloaders, such as GRUB or LILO. Your 
hardware manufacturer almost certainly will 
provide a bootloader, and | strongly suggest 
using it. If it does not support Linux, it 
usually is easier to adopt it than to port a 
different bootloader to your hardware. If 
you still choose to port the bootloader, Das 
U-Boot is a good choice, as are many others. 
The second missing piece is a Web-based 
graphical user interface (GUI), which most 
users come to expect from RGs. Most, if not 
all, currently available RGs have a Web inter- 
face written from scratch. However, it does 
not have to be like this anymore with the 
introduction of the X-Wrt Project, one of the 
components of the OpenWrt Project, which 
you almost certainly will want to look at if 
you are building an RG. OpenWrt is a Linux 
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distribution for the Linksys WRT routers (and 
not only those). 

Add to this a bit of init script writing, 
and you are done, except for the peripheral 
(802.11, ADSL and so on) drivers, which is a 
topic for a another article. 


Hardware 
Now to the real fun—choosing (or even 
better, designing) your hardware. MIPS is the 
most commonly used CPU architecture in 
these kinds of devices, with older devices 
using the MIPS32 4K cores, and newer 
devices probably using the MIPS32 24K 
cores. As MIPS 24K is about 3% slower than 
MIPS 4K at the same clock rate (because of 
the longer pipeline), it makes sense to use it 
only if you really need clock rates of about 
4O0OMHz. However, benchmarks show that 
networking devices are memory- and I/O- 
bound, not CPU-bound (after all, there is not 
much number crunching in forwarding pack- 
ets, with a few exceptions discussed later), so 
you probably will need these high clock rates 
only if you are going to use DDR RAM and 
not SDRAM (which is limited to 133MHz). 
Another option is the new (at least in the 
embedded world) multithreading (MT) tech- 
nology of the MIPS32 34K, which can have a 
configurable number of Virtual Processing 
Elements (VPEs) and Thread Contexts (TCs). 
Without going into too much detail, this 
technology is a more flexible equivalent of 
Intel Hyper-Threading, which helps the CPU 
exploit parallelism at the process and thread 
level (compared to parallelism at the instruc- 
tion level, which superscalar processors can 
exploit quite well). This can boost perfor- 
mance in some cases, especially when there 
are a few concurrent processes or threads 
that are I/O- or memory-bound, but bench- 
marks show that this may not always be the 
case. In addition, MT has an overhead—MIPS 
34K is slightly slower than MIPS 24K at the 
same clock rate, and the Linux kernel 
becomes slightly slower when compiled with 
symmetric multiprocessing (SMP) enabled. 
Another CPU you probably should consider 
is MIPS’ closest competitor—ARM. It is used 
less frequently than MIPS for these particular 
devices, mostly for historical reasons. ARM7 is 
very popular, small and has a low power core; 
however, it is clearly too slow for an RG. If you 
decide to go with ARM, you should consider 
one of the ARM9 or ARM10 families of pro- 
cessors, with the major differences, as far as 
an RG application is concerned, being perfor- 
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mance, power consumption and cost. 

RAM and Flash choice is obvious; you 
need at least 8MB of RAM and 2MB of Flash. 
However, as memory prices are dropping, | 
expect that future devices will have 64MB of 
RAM and 4MB of Flash. For the same reason, 
you probably will have to go with DDR and 
not SDRAM—as SDRAM actually becomes 
more expensive than DDR. 

Assuming you are going to use one of 
the reference design boards provided by a 
manufacturer (board design is beyond the 
scope of this article anyway), you are ready to 
test your brand-new RG. Unfortunately, you 
are in for a big surprise. Your CPU is going to 
choke at not so unreasonable bandwidths. 

One possible solution is to use a more 
powerful (and significantly more expensive) 
CPU, such as the Intel IXP or Freescale 
PowerQUICC network processors. For instance, 
the 533MHz IXP425 will have no problem sus- 
taining that kind of traffic. Unfortunately, in 
order to stay competitive, RG manufacturers 
cannot afford these high-end processors, so a 
more creative solution is required. 


Optimization 
This is the challenge of next-generation 
RGs—achieve high throughput using low- 
end hardware. One possible solution is to 
offload the whole data path of the RG to 
hardware, the way it works on high-end core 
routers, which usually employ a general-pur- 
pose CPU only for management and control. 
There are a few chips that can do this for the 
low-end RG market, such as the Realtek 
RTL8650B/RTL8651B, which can do routing, 
NAT and firewall in hardware. Of course, the 
implementation is limited compared to the 
Linux TCP/IP stack, but the hardware can be 
configured to trap the CPU in case it encoun- 
ters a packet it cannot handle, so that most 
of the packets will be forwarded from one 
interface to another without interrupting the 
CPU. However, this approach has a number 
of problems, the most serious one being the 
fact that the hardware TCP/IP stack is limited 
to a fixed interface (MII in case of RTL8650). 
It would be difficult and, in some cases, 
impossible, to connect other interfaces, such 
as 802.11, DOCSIS and xDSL, to that logic. 
Therefore, | believe that although this 
approach can work in some specific cases, 
it is a wrong idea in general. 

Typically, it is a good idea to keep to the 
software as much as possible, because it's 
easier to develop and debug. Another opti- 


mization approach is based on the observa- 
tion that RGs (and networking applications in 
general) are memory-bound, so it would be 
extremely beneficial to improve memory 
access. Let's separate data and code for the 
sake of this discussion. As far as the code is 
concerned, we want to keep it in cache as 
much as possible. Ideally, we want the whole 
critical path routines—that is, starting from 
one driver's receive function via the TCP/IP 
stack and to the other driver's send function 
to stay in cache all the time. This is not possi- 
ble with most embedded processors, which 
have only a 32K cache. However, it can be 
shown that the Linux 2.6 critical path—that 
is, functions used 95% of the time, under 
firewall and NAT configuration, including 
Ethernet drivers’ send and receive routines, 
can fit into a 64K cache, and there are 
embedded processors with 64K on the mar- 
ket. If your CPU does not have that much 
cache, but instead has scratchpad SRAM, you 
can modify the Linux linker script to place 
certain routines in the SRAM memory region. 

If you want to test the above observation 
(or calculate how much cache your particular 
application needs), use OProfile, which is a 
system-wide profiler for Linux that allows you 
to profile user-mode applications, kernel and 
drivers, and supports many embedded archi- 
tectures along with objdump (or any other 
utility) that will show you how much memory 
each routine requires. 

As for the data, it is absolutely necessary 
‘0 ensure that all network drivers follow 
zero-copy methodology, and it may be wise 
o place frequently accessed data structures 
in a scratchpad SRAM. 
Yet another optimization approach is the 
mixture of the above two—using profiler, find 
he most CPU-intensive pieces of code and 
offload this particular functionality to the 
hardware. As it turns out, the two most CPU- 
intensive tasks related to networking are IPSec 
and the UDP/TCP checksum calculation. It is 
very convenient (and not very surprising) that 
both have a well-defined architecture for 
hardware offloading. UDP and TCP checksum 
offloading is extremely beneficial, because if it 
is checked in the hardware on receive and cal- 
culated in the hardware on transmit, the CPU 
will never have to bring the whole packet into 
the cache, significantly reducing the number 
of memory accesses. IPSec, on the other hand, 
is less useful, as an RG is rarely an IPSec termi- 
nation point—usually IPSec (VPN) traffic is 
passed through and terminated on the PC. 


Another approach that | definitely do not 
recommend, but one being taken by some 
manufacturers because it is actually cheaper 
than the ones mentioned above, is to “opti- 
mize” Linux by creating various types of “fast 
paths”. For instance, if the L2 bridge perfor- 
mance is not satisfactory, it is possible to pass 
packets from one network driver directly to 
he other, eliminating at least one context 
switch and some other code, resulting in per- 
ormance gain. Although it will not work in 
general cases, it does work for an RG, where 
he manufacturer controls the whole system, 
including all the drivers and the kernel itself. 
The biggest problem with this approach is 
hat it actually cripples the stock Linux ker- 
nel, limiting functionality and introducing 
bugs. These modifications rarely are submit- 
ed to the Linux-kernel mailing list, and even 
if they were, they never would be accepted. 
But, they do go into some products you can 
ind in stores. 


Conclusion 

Using the steps described above, you should 
be able to build a Linux-based RG with rela- 
ively little effort. If performance becomes an 
issue, which almost certainly will be the case 
if you cannot use high-end processors, follow 
he optimizations guidelines outlined above. 
And, it’s always a good idea to run a profiler 
on your particular system to discover addi- 
ional bottlenecks. 

Although this article discusses RGs, most 
of the conclusions and guidelines are true for 
any embedded networking system. 

The issue of Linux optimization for RG 
systems actually leads to a much bigger and 
more controversial topic. There seems to be a 
significant communication problem between 
the Open Source community and embedded 
developers working for commercial compa- 
nies. On one hand, features added to the 
kernel sometimes hurt performance on small 
embedded systems. On the other hand, Linux 
improvements done by some companies do 
not always find their way back to the main 
kernel tree, often because they are not done 
properly. One good example of this miscom- 
munication is the 2.6 kernel itself, which 
included many important improvements for 
embedded systems, but suffered some perfor- 
mance degradation. As a result, a significant 
number of embedded systems still run the 2.4 
kernel. The reason for this miscommunication 
is probably the fact that semiconductor com- 
panies that usually do embedded software 


development find it hard to embrace the idea 
of open source, but it also may be due to the 
fact that the Open Source community is less 
interested in embedded systems, because they 
are harder to hack than a PC. | do believe that 
the first problem eventually will go away, as 
semiconductor companies understand how 
they can benefit from open source, and | try 
to do my share of explaining wherever | can. 
As for the second problem, one of the mes- 
sages of this article is that it’s easy and pretty 
cool to hack embedded systems, and you 
actually may have the hardware already.™ 


Alexander Sirotkin works for Metalink Broadband as a software 
architect. Metalink Ltd. (NASDAQ: MTLK) is a leading provider of 
high-performance wireless and wireline broadband communi- 
cation silicon solutions. Alexander has more than ten years’ 
experience in software, operating systems and networking, and 
he holds MSc and BSc degrees in Applied Statistics, Computer 
Science and Physics from Tel-Aviv University. 


Resources 


OProfile: oprofile.sourceforge.net 
uClinux: www.uclinux.org 
DENX: www.denx.de 


Das U-Boot: sourceforge.net/projects/ 
u-boot 


uClibc: www.uclibc.org 

BusyBox: busybox.net 

OpenWrt: openwrt.org 

X-Wrt: x-wrt.org 

Linux/MIPS: www.linux-mips.org 

ARM Linux: www.arm.linux.org.uk 

“An Introduction to Embedded Linux 
Development, Part 1”, by Richard A. Sevenich: 
www.linuxjournal.com/article/7848 

“An Introduction to Embedded Linux 


Development, Part 2”, by Richard A. Sevenich: 
www.linuxjournal.com/article/7911 


“An Introduction to Embedded Linux 
Development, Part 3” by Richard A Sevenich: 
www.linuxjournal.com/article/8001 
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Exploiting 64-Bit Linux 


How a big Mandelbrot can illustrate the advantage of using 64-bit Linux. STEVE MUNROE 


Wide availability of 64-bit systems offers an opportunity to simplify 
programming. By leveraging large virtual addresses and memory 
mapped files, we combine programming and file persistence into a 
single activity. This simplifies programming when persistence and shar- 
ing of large complex data structures are required. It also improves the 
efficiency of data access and sharing, as multiple programs can access 
data directly (operate in place) from the single real page copy. This 
eliminates the need to copy the data through multiple layers of buffer- 
ing. When all programs share data at the same virtual address, there 
are opportunities for the kernel to manage the memory map to avoid 
aliases and share MMU resources across applications. 

As a proof of concept, | have a small user library | call SPHDE 
(Shared Persistent Heap/Data Environment). This library manages 
a portion of the application's address space to provide blocks of 
persistent and shared storage. The library also provides some locking 
primitives and some “utility objects” that provide finer-grained 
storage allocation and indexing. 

As a demonstration, | wrote a gigapixel Mandelbrot (GTK2) GUI pro- 
gram. This program computes each point in the Mandelbrot set once and 
saves the resulting image as a quadtree in SPHDE storage. This quadtree 
representation can be shared and displayed by multiple instances of the 
program. Program instances can cooperate to fill in the next level of detail. 


Resources and Design 

System design and programming models are (initially) driven by the need 
to manage scarce resources. In the early days of programming, both 
address spaces and physical memory were small. |O Subsystems evolved to 
manage the buffering of serial media, such as tape and punch cards. Even 
the introduction of random access disk and virtual memory did not 
change the basic paradigm much. The programming paradigms we use 
and teach today have far outlived the original scarcity that caused them. 
The separation of programming from persistent data is a prime example. 
It creates inefficiencies in both programming effort and runtime execution. 


Opportunities 

With the prevalence of 64-bit microprocessors, the original (address- 
ing/memory) scarcity is gone. We can leverage virtual memory to 
access and share massive data structures directly using “operate in 
place” and “implicit persistence” patterns. This simplifies program- 
ming, because any data structure is implicitly persistent by where it is 
allocated. This improves operational efficiency, because it eliminates 
layers of buffer management and the OS has more opportunities to 
share a single physical copy. There are additional opportunities for 
improving address translation efficiency by eliminating address aliasing 
and exploiting hardware features, such as large pages. 


Some Alternatives Tried 
There have been attempts to unify programming and persistence before. 
Persistence frameworks abound, but they simply add layers of buffering 
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and data conversions to the top of the |O Subsystem stack. Operational 
efficiency is lost to obtain a modest reduction in programming effort. 

Alternatively, Single-Level Stores have been a research topic for 
years. Although the technology works (simplified programming and 
data efficiency), the requirement for specialized hardware or unique 
OS environments has limited acceptance. 

This picture changes again with widely available 64-bit micropro- 
cessors and standardized POSIX memory management APIs. The 
addition of a small runtime library allows adventurous programmers 
to use operate in place and implicit persistence across numerous 
OSes and 64-bit hardware platforms. This hybrid Single-Level Store 
approach allows for incremental exploration and adoption of a new 
programming paradigm. 


A High-Level Design 

Simple memory-mapped files is not enough. We need a design for how 
the address space and backing files are managed. The design should 
balance simplicity and efficiency in the management of the virtual 
(address space) and physical (real memory and disk space) resources. 

Here are some principles | believe are important to include in the 
design. The runtime should be simple to use and compatible with UNIX 
programming and POSIX APIs. It should use resources (address space) 
that are not normally used in existing applications. It should appear to the 
program like a large shared persistent heap (SPH). Access to data in the 
SPH should use standard C pointer semantics. Setup to access to data in 
the SPH should be minimum. The design evolved to the concepts of 
regions, stores, segment and blocks, all with power of two sizes. 

The region is simply the range of virtual addresses that the applica- 
tion wants the runtime to manage on its behalf. The region should be 
in a range of virtual addresses not normally used by the application. 
We want to ensure that the region does not interfere with the normal 
program usage of (local temporary) heap and dynamic libraries. On 
Linux, this means the area above the TASK_UNMAPPED_BASE and 
below the main stack. Leaving lots of room below the stack and above 
TASK_UNMAPPED_BASE still allows for the use of up to half the 
program's address space for SPH. 

Next, we need to choose a segment size. The segment is the unit 
size for allocating backing files and memory mapping those files into 
the region. This should be large enough so that we don’t do frequent 
kernel calls and small enough so that we don’t waste file space. 

A store is a directory that contains one or more files used to back 
SPH segments. A system can have multiple SPH stores by allocating 
the backing files in different directories. For now (also to keep things 
simple), a program can bind to only one region/store at a time. 

Finally, a block is the unit of space allocation within an SPH region. It is 
normally between a page and a segment in size, but that is not a hard 
limit. The only hard limit is that the block fits with the remaining space at 
the time. If the block is allocated to a portion of the SPH region that does 
not currently have a backing file allocated, the SPH runtime will implicitly 
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Listing 1. A List of Memory Blocks and Their Usage 


aseSEo address------ --size-- 
0, 0x400000000000, 1024KB 

Total in use 1024KB 

Total free OKB 

0, 0x400000100000, 1024KB 

1, 0x400000200000, 2048KB 

2, 0x400000400000, 4096KB 

3, 0x400000800000, 8192KB 

Total Uncommitted 15360KB 

0, 0x400001000000, 16384KB 

1, 0x400002000000, 32768KB 

2, 0x400004000000, 65536KB 

3, 0x400008000000, 131072KB 

4, 0x400010000000, 262144KB 

5, 0x400020000000, 524288KB 

6, 0x400040000000, 1048576KB 

7, 0x400080000000, 2097152KB 

8, 0x400100000000, 4194304KB 

9, 0x400200000000, 8388608KB 

10, 0x400400000000, 16777216KB 
11, 0x400800000000, 33554432KB 
12, 0x401000000000, 67108864KB 
13, 0x402000000000, 134217728KB 
14, 0x404000000000, 268435456KB 
15, 0x408000000000, 536870912KB 
16, 0x410000000000, 1073741824kB 
17, 0x420000000000, 2147483648KB 
18, 0x440000000000, 4294967296KB 
19, 0x480000000000, 8589934592KB 


20, 0x500000000000, 17179869184KB 
Total Region free 34359721984KB 
9, 0x400000000000, 16384KB 

Total Region used 16384KB 


create the file(s) in the store directory associated with the region. 

For overall simplicity, we use power of two sizes for blocks and 
segments. This allows a power of two buddy system to be used to 
track all aspects of SPH storage. The buddy system reduces fragmenta- 
tion and simplifies recombining smaller blocks into larger blocks when 
they are freed. 

We need some low-level utility functions to help manage the 
buddy lists. First, we need a simple heap manager to sub-allocate 
blocks for smaller control blocks and lists. The lists need to be ordered 
and will be searched frequently, so we use a simple binary tree. 
Algorithms for the heap manager and binary tree can be found in 
Donald E. Knuth’s The Art of Computer Programming (see Resources). 

Finally, we need a place to store and maintain this information. 
The anchor block is the first block in the region. The anchor block 
starts with a block header, which includes signature words (eye-catchers 
initialized to special values), type info, pointer to the start of the local 
free list (heap) and a set of pointers for the various binary trees that 
manage space in the SPH region. The remainder of the anchor block is 
initialized as free heap space. This internal heap is used to allocate 
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additional nodes required for the binary trees and any other internal 
control blocks needed to manage the SPH region. 

After an SPH region is initialized, we have mostly an empty region 
with one segment (backing file in the store directory) and one block 
(the anchor block) allocated from that segment. The various lists have 
been initialized to depict this state. For example, the region free and 
used lists contain entries for the unallocated (no backing files) and 
allocated (with backing files) portion of the region. An entry for the 
anchor block is added to the “used” list. The “uncommitted” list 
contains entries for the current unused portions of the first segment. 
And, the free list is empty. Now we are ready to allocate more blocks. 


The Lock Manager 

We choose a hash table-based lock manager using the virtual address 
as the lock ID. The addresses of data within SPH are unique, and 
active locks can be found quickly in a hash table. 

Storage for the lock tables has to be shared but not persistent, so 
we allocate IPC shared memory for that. This memory is initialized 
with a block header and associated heap, which is used to allocate 
storage for the lock hash table and lock lists. IPC semaphores also are 
allocated and used to block threads waiting on contended locks. 


Utility Objects 

So far, we have blocks of shared persistent storage and an address- 
based lock manager. Blocks are useful for storing large uniform arrays 
but are awkward to use for complex structures, such as link lists and 
trees. The SPH runtime includes utility objects that allocate and man- 
age blocks for finer-grained allocations and complex lists and trees. 

Utility objects all start with a block header and provision for an internal 
heap (using the same heap manager as the anchor block). The same signa- 
ture words are used, but each utility object has a unique type. The type 
values define a simple type system for runtime checks. The signature words 
and the fact that blocks are all powers of two sizes simplifies finding the 
block header for any utility object from any address within a block. This is 
the trick supporting the new-near allocation scheme discussed later. 

The simplest utility object is SPHSimpleStack. The SimpleHeap is 
simply a block header and internal heap. A CompoundHeap is a heap 
manager that allocates SimpleHeaps. The block header links multiple 
CompoundHeap blocks together to form an expandable 
CompoundHeap. This “heap of heaps” structure, combined with the 
“new-near” mechanism is useful for maintaining locality of reference 
for large complex list and tree structures. 

The CompoundHeap is a framework (think superclass) for the 
SPHStringBTree, SPHIndex and SPHContext utility objects. The 
SimpleHeap is the framework for the BTree nodes internal to 
SPHStringBTree and SPHIndex. An SPHStringBTree maps a string 
(name) to an address. An SPHIndex maps an arbitrary binary value to 
an address. An SPHContext defines a two-way mapping between 
one or more strings and an address. These utility objects are useful 
for creating naming structures and content-addressable memories. 


Handling Gigapixel Images 

| needed a test for Shared Persistent Heap runtime and thought 
storing and processing large images would be interesting. In a 
previous personal project, | implemented a fast Mandelbrot Fractal 
(see Resources) display based on breaking the image in quadrants 
and storing the image in a quadtree structure. The Mandelbrot set 
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SPHDE Mandelbrot 
File Edit GigaPixel View Help 


Gen lime 0.0310005 Level 9 
compute ( 2.0000000000, 2.0000000000) (2.0000000000,2.0000000000) 


Figure 1. Mandelbrot Set with Quadtrees Outlined 


is interesting, because it is “self-similar” and shows detail and any 
zoom factor. 

This program could pan and zoom over color renditions of the 
Mandelbrot set where most of the image was pre-computed and 
stored in the quadtree. The algorithm is incremental, so detail is com- 
puted and added to the quadtree as needed (for the current display). 

At the time, | had no good way to store the resulting quadtree 
for later use. | did write recursive streaming code to write/read a 
flattened representation of the quadtree to/from a file. But, as the 
quadtree image accumulated detail, it slowed down noticeably. 
Also, | ended up with multiple files with pre-computed details of 
different areas of the Mandelbrot set, but had no good way to 
merge them in a single high-resolution image. At this point, the 
Mandelbrot Project was set aside. 

Later, when | was working on the SPHDE Project and was trying to 
think of a good demo, | remembered the quadtree Mandelbrot 
Project. The hard part was converting the original Mandelbrot program 
from Borland OWL to Linux and GTK2. The actual conversion to use 
SPHDE was much easier. 

First, | added SPHJoinRegion and SPHCleanUp calls to the entry 
and exit of main. Then, | added code to handle first-time setup. This 
involved allocating a control block to anchor the quadtree and create 
an expanding SPHCompoundHeap to manage storage of the 
quadtree. Subsequent use of the program needs to obtain only the 
address of this control. This pointer can be stored and retrieved from a 
free slot in the anchor block header. 

The next step was to change the quadtree algorithm to use SPHDE 
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SPHDE Mandelbrot 
File Edit 


GigaPixel View Help 


compute ( 0.7500305176, 0.0245208740) { .0.7495422363, 0.0240325928) 


Figure 2. Zoom 8192X with Quadtrees Outlined 


storage. This is only slightly complicated by a desire to maintain good 
locality of reference within the quadtree itself. The SPHCompoundHeap 
allocates SPHSimpleHeaps from which quadtree nodes are allocated. 
Allocating adjacent quadtree nodes from the same SPHSimpleHeap 
ensures physically locality in memory and the backing file. This minimizes 
the number of pages touched to display the whole Mandelbrot set (the 
topmost part of the quadtree) or zoom to any part of the set. 

The simple call: 


node = (TQuadTree*) SPHSimpleHeapNearAlloc (near, sizeof (TQuadTree)); 


always will attempt to allocate from the SimpleHeap containing the 
nearest existing quadtree node. If this allocation fails, we either need 
to find another SimpleHeap with free space or call 
SPHCompoundHeapAlloc to allocate another SimpleHeap: 


if (!node) 
{ 
templ = SASCompoundHeapAlloc (QuadBlockPtr->compoundHeap) ; 
if (temp1) 
{ 
node = (TQuadTree *) 
SASSimpleHeapAlloc (temp1, sizeof (TQuadTree)); 
lastAlloc = temp1; 


A simple cache of recently allocated SimpleHeaps and SimpleHeaps 
where nodes were recently freed served to keep the quadtree relatively 
dense within the SPHCompoundHeap managed storage. All this logic 
is contained in 100 lines of code. 

Finally, we need to ensure that multiple instances of the program 
can share and update the quadtree safely. This requires adding 
SPHLock/SPHUnlock calls around any code with the potential to modify a 
quadtree node. There are only four such locations in the current algorithm. 
The utility objects (SimpleHeap, CompondHeap and so on) use appropriate 
locks internally to ensure consistent behavior in a shared environment. 


Conclusion 

The result is a program that can compute and store a gigapixel or 
arger image. Generating a gigapixel (32768x32768) Mandelbrot 
image takes about five minutes (Apple G5 dual-2.3GHz PowerPC). 
Once the gigapixel quadtree is generated, exiting and restarting is 
nearly instantaneous (24 milliseconds to display a 512x512 pixel 
image). Panning and zooming around the pre-computed quadtree 
is also fast (12-20 milliseconds). Zooming to a depth beyond the 
current pre-computed set will slow down (200-500 milliseconds) due 
o the heavy computation required. Once an area is computed and 
stored, subsequent displays are fast (12-20 milliseconds) again. 

The SPHDE runtime and Mandelbrot demo program run on 
both 32-bit and 64-bit Linux, including the PowerPC, i386 and 
x86_64 platforms. 32-bit systems can support a 1-2GB region, 
which is large enough to store a gigapixel Mandelbrot quadtree 
(~313MB). Larger (terapixel) images will require the large regions 
allowed by 64-bit systems. With recent Linux kernels, PowerPC 64 
can support regions in the 8TB range. The X86_64 platform 
supports a 128TB address space and a 647TB region. This is more 
than enough for some interesting projects.™ 


Steve Munroe is currently an architect covering toolchain (GCC, GLIBC, binutils, GDB and related 
performance collection tools) issues in IBM's Linux Technology Center. Steve is an active contributor 
to open-source software, porting the GNU C library (GLIBC) to 64-bit PowerPC. Steve is an expert 

in single-level storage, large address space architectures and Java, with an emphasis on object 
persistence and business applications. He coauthored (with Steven Halter) the book Enterprise Java 
Performance. He also was a performance architect for IBM's San Francisco Project and authored a 
Java workload from which he helped SPEC (Standard Performance Evaluation Corporation) create 
SPECjbb2000, an industry-standard workload for Java server performance. 


Resources 


Enterprise Java Performance, Chapter 14, Steve L. Halter and Steven 
J. Munroe, Prentice Hall 2001, ISBN 0-13-017296-0. 


The Art of Computer Programming, Volume 1, Fundamental 
Algorithms, 3rd ed., Donald E. Knuth, Addison Wesley 1997, 
ISBN 0-201-89683-4. 

SASOS: liinwww.ira.uka.de/csbib/Os/sasos.html 


Mandelbrot Set: en.wikipedia.org/wiki/Mandelbrot_Set 


Quadtree: en.wikipedia.org/wiki/Quadtree 
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Take a Peek at Some of the 
Freshest Projects Around 


Initial screenshots of LMMS will give you a hint of FruityLoops and the 
like, but in reality, LMMS takes a completely different tack. LMMS 
takes on a micro-managed approach, and it probably will appeal to 
control freaks who want to generate their own samples or instruments 
from scratch. With the ability to layer instruments and tracks, you can 
create whole songs from the ground up. 

Unlike most compositional software under Linux, LMMS takes an 
alternative approach to midi for many circumstances, by synthesizing 
through wave instead. This sidesteps the often annoying process of 


j.S.Bach-Preludium_and Fuge A-Minor - LMMS 0.2.1 
Project Edit Settings Help 


LMMS—the Music Suite for the Ultimate Control Freak 
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JOHN KNIGHT 


making midi work under Linux, although midi still can be used and is 
very much a part of LMMS. By tweaking all the knobs and available 
options, you can generate entirely original sounds and use them in 
any way you see fit. 

| managed to generate some brilliant effects and samples for my 
own musical projects. This likely will appeal most to dance-music DJs 
and also to fans of Nine Inch Nails or the more recent Massive Attack- 
style music for those who want unique samples, or it will appeal to 
fans of retro 1980s synthesized sound for those who want to make 
whole orchestrations. 
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Installation is a breeze, as LMMS resides in many distributions’ 
repositories. Under Debian-based distributions, all | had to do was 
a simple: 


# apt-get install lmms 


For those who can’t find a binary for their system, however, 
a simple: 


$ ./configure 
$ make 
(as root or sudo) 


# make install 


with the source is all that is required. Thankfully, LMMS doesn't have 
too many strange dependencies, so it should compile right off the bat 
for most systems. 

Once you dive into the world of LMMS, a steep learning curve 
presents itself. Some tutorials and demos are included, but these 
will leave most users in the dark on many issues (but they are 
worth examining as they do uncover the uses of many GUI items). 
The demos are indeed worth a look and show off the capabilities 
of the program. The demos include Bach’s “Preludium and Fuge in 
A Minor”, which is impressive when you watch the keys played in 
real time under each instrument's settings, and other cool demos 
are included that range from general dance music to the occasional 
more atmospheric piece. 

The best tip for exploring LMMS for the first time is to click on the 
yellow star on the left whose tab is titled My presets. From here, you 
can play around with some already-tweaked instrument settings and 
various GUI buttons and sound settings until you come up with some- 
thing that sounds interesting and the interface becomes more familiar. 
Also, be sure to look at the green musical note above the yellow star, 
titled My samples. This contains the base instruments to begin with 
and offers a large variety, including (but not limited to) some impres- 
sive drum samples, Latin guitars, as well as some amazing string, choir 
and atmospheric effects. 


Despite all of the features and demos, however, it’s worth visiting 
the LMMS Wiki. Documentation is still lacking for this project, but 
hopefully this improves in the future, particularly in relation to chord 
compositions and note editing. Documentation issues aside, the pro- 
gram is quite stable at the moment and can at least be used as a good 
beat sequencer. If you can figure out the interface for instrument 
composition, whole tracks also will be at your fingertips. 

Some other impressive features are JACK and LADSPA support, but 
best of all is the ability to mix an instrument in surround as opposed to 
just stereo. When | asked lead developer Tobias Doerffel about the 
development status, he mentioned that LMMS is still rather unstable 
on 64-bit platforms, but after this is fixed, it will be in a quite usable 
state. He also wants to include a mixer that mixes all of the tracks as 
well as improve the available JACK support. 

Currently, this is fairly mature beta software that looks extremely 
promising, and it is lacking only decent documentation—definitely 
worth a look for any home musician. 

For more information on LMMS, see: 


™ Home Page: Imms.sourceforge.net 


® Download Page: Imms.sourceforge.net/download.php 
@ Wiki: wiki.mindrules.net 


inotail—Inotify Enhanced Tail 

For all you system administrators out there keeping track of important 
log files with tail, this is definitely worth a look. When using the follow 
mode (tail -f filename), tail re-reads a file once a second by 
default. inotail takes a different approach by making use of a newer 
kernel feature, the inotify API. Instead of a clumsy cyclical update 
based purely on time, inotail listens to special events sent by the 
kernel using the new API. 

After testing inotail, | was happy to see its results. | simply took a 
text file and read it with the command inotail -f test.txt. With 
the text file, | added lines one at a time and saved the document each 
time as | added a line. As soon as | pressed Save, the inotail screen 
output instantly updated without a hitch. Okay, so it’s not exactly 
going to impress your mates, but system administrators probably are 
going to find an instant use for this far more elegant approach to 
keeping an eye on file updates. 

I'm afraid that you still will have to compile this one from source, but 
it isn't hard. According to developer Tobiad Klauser’s Web site, inotail 
should be in the Debian repository soon, but it isn’t there at the time 
of this writing. To compile and install inotail, simply extract the tarball 
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(available on the main page of the Web site), and go to the new 
directory. Enter the commands: 


$ make 
(as root or sudo) 


# make install 


As with most source compilations, this places the executable in 
/usr/local/bin by default. If you would rather place it in /usr/bin, enter 
the command: 


# make prefix=/usr install 


Karmen—for a Clean and Minimalist Environment 


70 | august 2007 www.linuxjournal.com 


The only real factor hampering this project for now is time. inotail 
requires at least a 2.6.13 kernel, which still is fairly young for many 
system administrators who tend to use somewhat older and more 
mature distributions than the rest of us. After a year or so has passed, 
when inotail has been accepted into the Debian archive and adminis- 
trators have upgraded their distributions, inotail should be finding its 
way into many an admin's toolbox. 

Home Page: distanz.ch/inotail 


Karmen 

For those of you who like minimalist window managers, Karmen may 
be your future choice. Designed to “just work”, it has no dependen- 
cies except Xlib and no configuration file to fiddle around with. 
According to the README file, these are its main goals for full release: 
@ Intuitive, efficient window management. 


® Provide a high-quality look and feel. 


® Standards compliance (ICCCM and EWMH/NetWM). 


XMNS 
MPlayer 

xterm 

Shell - Kansole 


@ Work well as a standalone window manager. 
@ Work well with other desktop utilities and environments. 
™@ Focus on window management and let other tools do the rest. 


Head to the main Web site to grab the latest tarball. Indeed, 
installing it was a cinch. Doing a simple: 


$ ./configure 

$ make 

(as root or sudo) 
# make install 


was all that was required, and as it says on the tin, there are no weird 
little dependencies to get in the way. It also happens to be quick and 
sleek, and each time | turned my head back around to look at the 
monitor, configure, make or make install was already done! 

Once you have the Karmen desktop running, moving windows 
around is nice and familiar in a KDE/GNOME/Windows way, with 
clicking to focus and window resizing working in the intuitive way 
users have come to expect. Rather than the all-too-common annoyance 
of having to click on a window’s titlebar to change focus that plagues 


many lightweight window managers, Karmen lets you switch window 
focus by clicking within a window's body. Maximizing is a different 
affair though; whereas two obvious minus and close buttons sit at the 
top right of a window, maximizing requires that you double-click the 
titlebar—not immediately obvious. For a list of windows (whether 
active or minimized), right-click on the desktop, and a new menu 
appears, allowing you to bring to focus any windows currently running. 

Another interesting note: Karmen seems to be quite a keyboard- 
driven window manager. For instance, to tell a window to stay on top 
of others, click the titlebar with the Shift key held down. Shift-clicking 
again disables the stay-on-top property of the window. A welcome 
addition to minimalist window managers, pressing the familiar old 
Alt-Tab cycles between windows. In fact, most basic GUI functions 
can be performed via the keyboard, generally by using a combination 
of Alt and another key. 

However, Karmen is quite young in its development, with some 
strong limitations in its current form. Still lacking is any kind of menu 
for major functions, such as logging out or choosing a simple xterm. 
Indeed, Karmen still requires that you kill its process manually to exit— 
obviously something that will be changed in the future, but a definite 
indicator of a project that's still in its infancy. Lacking too are startup 
scripts to start the window manager cleanly; you either have to make 
your own script or edit .xinitrc. For those who can't be bothered with 


Don't complicate a simple task 


Keep basic tasks just that with handheld, stationary and 


vehicle-mounted wireless data collection from AML. While others 


are busy reinventing the wheel, we're keeping things simple, 


from our products to our personal service. Visit us at www.amitd.com 


M7 100 Wireless 


or call 1.800.648.4452 for a real live person 


NEVER COMPLICATED” 


Family 


www.linuxjournal.com august 2007 | 71 


Window 1 


File Window Layer Ught Assembly Graph Shape Arrow Cel 


Text Ororsl Bond Atom o thee Group Molecule Cluster 


Console 


GAMGI—for the mad scientist. Can you believe that thing is a heroin molecule? 


this, you also could start Karmen from the command line, but you 
have to add & xterm specifically after the command. Thankfully, these 
topics are touched upon in a nice man page—a professional touch at 
this early stage. Also, for now, Karmen is a strictly one-desktop affair, 
but hopefully this also will change. 

All limitations aside, however, developer Johan Veenhuizen’s 
approach seems to be one of working on individual sections cleanly 
and then moving on. The coding is indeed very clean and stable— 
although in its infancy—and looks like it will be a snazzy little desktop 
once finished. | reckon Karmen definitely will find a home with many 
niche users once it matures. 

Karmen: karmen.sourceforge.net 


GAMGI—General Atomistic Modeling Interface 
For this last project, | thought I'd throw in something for the mad scien- 
tists out there. According to the home page, GAMGI’s goals are to pro- 
vide the community with a free software package to construct, view and 
analyze atomic structures, and to make it as powerful and as simple to 
use as possible. GAMGI's developers also are probably the most hard- 
core free-software advocates I’ve seen since Stallman himself. Scattered 
throughout the installation instructions and user manuals are loads of 
exhortations to harness the power of source and compile it yourself, as 
well as many other passionate free-software-related endeavors. 

Here are GAMGI's stated aims from these obviously passionate 
developers: 


GAMGI aims to be useful for: 1) the scientific community work- 
ing in atomistic modeling, which needs a graphic interface to 
build and analyze atomic structures; 2) the scientific community 
at large, which needs a graphic interface to study atomic struc- 
tures and to prepare images for presentations; 3) teaching the 
atomic structure of matter in schools and universities, even invit- 
ing students to run GAMGI at home; 4) science promotion, in 
exhibitions and science museums. 


Installation can be a little tricky for the newbie, but anyone com- 
fortable with things such as make’s role or symlinking should be okay. 
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GAMGI is designed to be as standardized as possible in its require- 
ments, and it should work with any standard X11 installation, though 
it does have one or two esoteric library requirements. Glib and GTK 
should be fine with most systems, as well as FreeType, but you may 
have to seek out and install Expat and Gtkglarea. 

For those seeking a simple binary, an RPM is usually provided at 
GAMGIl's download page. The RPM is designed to be LSB-compliant 
and should convert easily with a utility such as alien for Debian-based 
systems. At the time of this writing, however, the link was taken 
down, so please check the site again at a later date or bug the 
developers. For those seeking to compile from source (keeping the 
developers happy), a few more steps than your run-of-the-mill tarball 
need to be taken. First, it's easiest if GAMGI is extracted and compiled 
somewhere public, such as /usr/local/src, as you probably will want 
to symlink to the compiled binary later. Grab the tarball from the 
given link and extract it. 

Once extracted, simply open a console and make your way to the 
src directory under the new folder. Now run the command make, and 
if all goes well, GAMGI should compile successfully. If make returns 
errors, check that you have all of the needed libraries installed. If they 
are, your shared libraries may be installed to another path than what 
GAMGIl's make script is expecting. Edit the make_local file in the src 
directory and continue. Once compilation has completed, you can run 
the gamgi binary from the src directory, or symlink to it for easier 
access with the command: 


# In -s /usr/local/src/gamgi-(version)/src/gamgi 


> /usr/bin/gamgi 


Once all that nasty stuff is out of the way, the weird science can 
begin. Some interesting templates and tutorials have been included in 
GAMGl's dat directory. From the menu, choose File—Import and 
choose an atomic structure at random. Ever wondered what adrenaline 
ooks like? Try molecule/drugs/adrenaline.xml. Presto! An adrenaline 
molecule appears in the main black window. To play around with the 
view, press and hold the left, middle or right mouse buttons and move 
he mouse around. The left button controls rotation, the middle but- 
ons controls the X and Y axes, and the right button controls the 
zoom function. You have to admit, it is pretty cool to play with! 

To explore further, there are a bunch of other menus related to all 
hings atomic, but | haven't the foggiest idea how to use any of them (or 
what they even do). | think this is best left to the atomic scientists. Check 
he doc directory for further instructions, and have a read through the 
well-constructed HTML manual pages, or head over to the GAMGI home 
page and converse with fellow white-coated boffins. Either way, I’m sure 
‘Il sleep easy tonight now that | know what a propane molecule looks 
ike and have confirmed that | am, indeed, a geek! 

For more information, see: 


™ Home page: www.gamgi.org 
@ Tarball: www.gamgi.org/src/gamgi-all-0.12.5.tar.gz 


@ Wiki: www.gamgi.org/wikil 


John Knight is a 23-year-old drumming, climbing and music-obsessed maniac who usually can 
be found squeezing Audacity to within an inch of its life. 
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Building Firefox Extensions 


How to create your own Firefox extension. JUSTIN HUFF 


Like many Linux Journal readers, Firefox is my browser of choice. 
One of its core strengths is the number of available extensions. Early 
extensions were focused around merely changing the look of the 
browser; however, in the past few years, extensions have been used to 
provide a very rich user experience while straddling the line between 
desktop and Web applications. 

In this article, | explain how easy it is to extend Firefox by building an 
extension that integrates with a photo editing API provided by Picnik. 


Setting Up the Environment 

The first thing a new extension developer should do is set up a devel- 
opment profile. Although you can do extension development using 
your normal Firefox profile, it often is easier to create a new profile 
dedicated to development. First, start Firefox's profile manager: 


$ firefox -ProfileManager 


Next, click the Create Profile button. Once the wizard is loaded, click 
Next to get started. At this point, you should see a window similar to the 
one shown in Figure 1. Enter a name for your new profile (| used dev). 
Make sure you write down the path to the folder where your profile will 
be stored before clicking Finish. You'll be using that path later. 


reate Profile Wizard Now tha you 
Completing the Create Protite Wizard have a dedicated 
profile, you 
f you create several profiles you can tel them apart by the profile names. Z 
You may use the name provided here of use one of your own: should instal 


Enter new profile name: 
[aed 


some extensions 
that make devel- 
opment easier. 
The first one you 
should install is 
the Extension 
Developer. This 
is a compilation 
of several handy 
extensions—all 
designed to make 
developers’ lives easier. See Resources for several other handy exten- 
sions. | highly recommend that you install all of them. 


Your user settings. preferences, bookmarks and mal will be stored in: 
Mome/finull/ mozMafretcxsmnasppm.dev 


Choose Folder 


Click Finish to create this new profle 


X Cancet 


@ Bock Finish 


Figure 1. Creating the Profile Named dev 


Your First Extension 
At this point, you're ready to start your first extension. Nearly all 
extensions start with the same basic boilerplate code, so the same 
person who made the Extension Developer put together the Firefox 
Extension Wizard to automate this part of the process. You can find its 
URL in the Resources for this article. 

Most of the required fields should make sense. The main one of 
note is the Extension ID. This is used to identify the extension uniquely 
for updates and other purposes. In the past, standard practice was to 
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Create Extension 


d's Me: Page 


Figure 2. Navigating the Firefox/Thunderbird Extension Wizard 


use a GUID (Globally Unique Identifier). Most developers recently have 
switched to a format that bears resemblance to an e-mail address. 
For this example, | used extension@linuxjournal.com. | also selected 
the option to create a context (right-click) menu. Figure 2 shows how 
| filled in the rest of the fields. 

Once you are satisfied with your choices, click the Create 
Extension button. After a few seconds, your browser should prompt 
you to download a zip file. Go ahead and extract it: 


$ unzip linuxjournal.zip 
Archive: linuxjournal.zip 
inflating: Linuxjournal/install.rdf 
inflating: Linuxjournal/chrome.manifest 
inflating: lLinuxjournal/readme. txt 


inflating: Linuxjournal/content/firefoxOverlay.xul 
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inflating: Linuxjournal/content/overlay.js 

inflating: Linuxjournal/skin/overlay.css 

inflating: lLinuxjournal/locale/en-US/linuxjournal.dtd 
inflating: Linuxjournal/locale/en-US/linuxjournal.properties 
inflating: lLinuxjournal/config_build.sh 


inflating: lLinuxjournal/build.sh 


Before going into the purpose of all those files, you should install 
it to see what the auto-generated extension actually looks like. 
Firefox can use extensions installed in two ways. The normal installa- 
tion method involves opening the extension’s .xpi file in Firefox. This 
is the way most extensions are distributed and installed. The other 
method is to create a pointer file that tells Firefox where to find your 
extension’s files. With this method, you don’t have to re-install the 
extension every time you want to test a change; all you have to do 
is create the pointer file: 


$ cd linuxjournal 
$ pwd > ~jjhuff/.mozilla/firefox/lhn85ppm.dev/extensions/ 


™extension\@Linuxjournal.com 


Of course, you'll want to replace ~jjhuff/.mozilla/firefox/lhn85ppm.dev 
with your Firefox development profile directory. 
Now, go ahead and start up Firefox using your development profile: 


$ firefox -P dev 


First, check to see that the extension is installed. Select 
Tools—Add-ons, and verify that LinuxJournal 1.0 is listed. You should 
see a window like the one shown in Figure 3. While you have the 
Tools menu open, you probably noticed the new (and red) menu item 
(Figure 4). Go ahead and right-click in the browser window. You 
should see a menu similar to the one shown in Figure 5. If everything 
looks right, your extension is installed properly. 


Innards 

Before modifying the generated code, you should understand how all 
the pieces interact. The main file for the extension is install.rdf. It spec- 
ifies the extension’s name, ID and version. The install.rdf file also con- 
tains a list of all the compatible applications and their versions. In this 
example, we specify a single application with an ID of {ec8030f7-c20a- 
464f-9b0e-13a3a9e97384}, which is the ID for Firefox. We also specify 
that we're compatible with Firefox versions 1.5 through 2.0. 

The second file of interest is chrome.manifest, which tells Firefox 
what to expect inside the extension. The manifest also includes a list 
of overlays. (| explain overlays later in this article.) 

Most extensions are organized into several directories. The 
content directory typically contains the bulk of your extensions 
Ul and logic. The skin directory is where CSS and any graphics 
live. Finally, locale is for locale-specific files, such as translations. 

(I discuss localization later in this article.) 


Chrome 

The user interface for Firefox (as well as some other Mozilla pro- 
jects) is implemented in a file format known as XML User Interface 
Language (XUL) combined with JavaScript. Collectively, this is 
known as Chrome. If you installed the Chrome List extension, you 
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Figure 3. The LinuxJournal extension is installed and ready to go. 
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Figure 4. The extension can create a custom menu item under Tools. 
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Figure 5. You also can create a 
custom right-click menu item. 


overlay 
chrome: //browser/content/browser.xul 


>chrome://Linuxjournal/content/firefoxOverlay.xul 


This line specifies that firefoxOverlay.xul should be overlaid on top 
of browser.xul. If you add additional overlay files, or if you want to 
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modify other parts of the application, you need to add more lines to 
chrome.manifest. 

Let's take a look at how the extension adds a new item to the 
context menu. First, open chrome://browser/content/browser.xul in 
the Chrome Browser, and search for contentAreaContextMenu. 
The second hit should look similar to this: 


<popup id="contentAreaContextMenu" ... > 


<menuitem id="context-stop" 
label="&stopCmd.label;" 
accesskey="&stopCmd.accesskey;" 


command="Browser :Stop"/> 
<menuseparator id="context-sep-stop"/> 
</popup> 


Now, open up firefoxOverlay.xul from your extension. You should 
see a block that looks like this: 


<popup id="contentAreaContextMenu"> 
<menuitem id="context-linuxjournal" 
label="&LlinuxjournalContext.label;" 
accesskey="&1LinuxjournalContext.accesskey;" 
insertafter="context-stop" 
oncommand="1inuxjournal.onMenuItemCommand (event) "/> 


</popup> 


When the overlay is loaded, the browser searches its existing Chrome 
for an element with the ID of contentAreaContextMenu and merges in 
the XUL from the overlay. It will end up with something like this: 


<popup id="contentAreaContextMenu" ... > 


<menuitem id="context-stop" 
label="&stopCmd.label;" 
accesskey="&stopCmd.accesskey;" 


command="Browser:Stop"/> 


<menuitem id="context-linuxjournal" 
label="&LinuxjournalContext.label;" 
accesskey="&LinuxjournalContext.accesskey;" 
insertafter="context-stop" 


oncommand="Linuxjournal.onMenuItemCommand (event) "/> 

<menuseparator id="context-sep-stop"/> 
</popup> 

When this menu is rendered, our menu item appears along 
with the normal context menu items. In addition, we specified the 
insertafter attribute to tell the browser that we want our menu item to 
appear after the context-stop menu item. 
Localization 


The Chrome system builds on existing technologies to support localizing 
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Listing 1. picnik.xul 
<?xml version="1.0"?> 
<!DOCTYPE picnik SYSTEM "chrome://picnik/locale/picnik.dtd"> 


<overlay id="picnik-overlay" 
xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul" 
xmins:html="http://www.w3.org/1999/xhtm1"> 


<script type="application/x-javascript" 
src="chrome://picnik/content/common.js"/> 
<script type="application/x-javascript" 


src="chrome://picnik/content/contextmenu. js"/> 


<popup id="contentAreaContextMenu"> 
<menuitem class="menuitem-iconic" 
id="picnik-ctx-edit" 
insertafter='context-viewimage' 
label="&picnik.edit_picture;" 
oncommand="picnikContextMenu. editImage() ;" 
image="chrome: //picnik/content/picnik_16x16.png"/> 
</popup> 
</overlay> 


the UI easily. One major part of this is the ability to store strings 
separately from the UI itself in a DTD (Document Type Definition) file. 
In our code, firefoxOverlay.xul references that DTD with the line: 


<!DOCTYPE overlay SYSTEM "chrome: //1inuxjournal/locale/1linuxjournal.dtd"> 


Chrome URLs for locale are special, because the browser automatically 
expands them to reference the proper location in the extension. For 
example, Firefox automatically expands chrome://linuxjournal/locale/ 
linuxjournal.dtd to chrome://linuxjournal/locale/en-US/inuxjournal.dtd for 
US English speakers. 

The DTD is used to define new XML entities, which can be 
thought of as macros. Our DTD contains: 


<!ENTITY linuxjournal.label "Your localized menuitem"> 
<!ENTITY LinuxjournalContext.label "Your Menuitem"> 


<!ENTITY linuxjournalContext.accesskey "Y"> 
These are referenced in the XUL by prefixing them with an &, as in: 


<menuitem id="context-linuxjournal" 
label="&LinuxjournalContext.label;" 


accesskey="&1LinuxjournalContext. accesskey; " 


This separation of strings from the Ul can be awkward at first, but 
it has other advantages beyond localization. For example, if you make 
a spelling error in something that appears in multiple places, you have 
to fix it only once. 


Code 
The actual code behind extensions is written in JavaScript, which makes 
writing them within reach of many seasoned Web developers. JavaScript 


Listing 2. contextmenu.js 


var picnikContextMenu = { 
onLoad: function() 
{ 
// Attach the showContextMenu function 
// to the context menu 
var e = document.getElementById("contentAreaContextMenu") 
hie )) 
e.addEventListener("popupshowing", function(ev) { 
picnikContextMenu.showContextMenu(ev); }, false); 


}, 


// Called right before the context menu 
// popup is shown 
showContextMenu: function(event) 
{ 
if( gContextMenu ) 
{ 
var edit_picture = document.getElementById("picnik-ctx-edit") ; 
if( edit_picture ) 
edit_picture. hidden = ! (gContextMenu.onImage | 
gContextMenu. hasBGImage) ; 


if( gContextMenu.onImage ) 

this.imageURL = gContextMenu. imageURL 
else if( gContextMenu.hasBGImage ) 

this. imageURL = gContextMenu.bgImageURL 
else 


this.imageURL = ''; 


// Called if the user clicks the ‘edit’ 
// menu item 
editImage: function() 
{ 
var url = picnikCommon.baseURL + "?import="_+ escape(this. imageURL) 


gBrowser.selectedTab = gBrowser.addTab(url) ; 


be 


window. addEventListener("load", picnikContextMenu.onLoad, false); 


also makes it easy for extensions to be cross-platform with very little 
work. In order for the code actually to be loaded, it must be referenced 
in an overlay XUL file. In our case, the following does the trick: 


<script src="overlay.js"/> 


One of the most important points to remember when writing 
extensions is that JavaScript's global namespace is shared between 
all the extensions as well as the core browser code. This means that 
developers need to use techniques to prevent name clashes. One 
simple method is to add a unique prefix to all of your variables and 
functions. The preferred method is to create an unnamed object that 
contains all of your variables and functions. Taking a look at the 
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Listing 3. Expanded Version of the picnik.xul File 
<?xml version="1.0"?> 
<!DOCTYPE picnik SYSTEM "chrome: //picnik/locale/picnik.dtd"> 


<?xml-stylesheet href="chrome://picnik/content/toolbar.css" 
type="text/css"?> 


<overlay id="picnik-overlay" 


xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul" 
xmins:html="http://www.w3.org/1999/xhtm1"> 


<script type="application/x-javascript" 
src="chrome://picnik/content/common. js"/> 
<script type="application/x-javascript" 
src="chrome://picnik/content/contextmenu. js"/> 
<script type="application/x-javascript" 


src="chrome://picnik/content/screengrab.js"/> 


<popup id="contentAreaContextMenu"> 
<menuitem class="menuitem-iconic" 
id="picnik-ctx-edit" 
insertafter='context-viewimage' 
label="&picnik.edit_picture;" 
oncommand="picnikContextMenu. editImage() ;" 
image="chrome://picnik/content/picnik_16x16.png"/> 


<menu class="menu-iconic" id="picnik-ctx-grab" 

insertafter='context-sendpage' 

label="&picnik.menu;" 

image="chrome://picnik/content/picnik_16x16.png"> 

<menupopup> 

<menuitem Label="&picnik.grab_visible;" 
oncommand="picnikScreenGrab.grabVisible() ;"/> 

<menuitem Label="&picnik.grab_full;" 
oncommand="picnikScreenGrab.grabFull() ;"/> 


</menupopup> 


auto-generated extension we see: 


var linuxjournal = { 


onLoad: function() { 

}, 

showContextMenu: function(event) { 
Py 

onMenultemCommand: function(e) { 
bs 


ie 
window.addEventListener("load", function(e) 
‘{ linuxjournal.onLoad(e); }, false); 


Using this technique, our extension has exactly one entry in the global 
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</menu> 


</popup> 


<menu id="tools-menu"> 
<menupopup id="menu_ToolsPopup"> 
<menu class="menu-iconic" id="picnik-tool-menu" 
label="&picnik.menu;" 
image="chrome://picnik/content/picnik_16x16.png"> 
<menupopup> 
<menuitem label="&picnik.grab_visible;" 
oncommand="picnikScreenGrab. grabVisible() ;"/> 
<menuitem label="&picnik.grab_full;" 
oncommand="picnikScreenGrab. grabFull() ;"/> 
</menupopup> 
</menu> 
</menupopup> 


</menu> 


<toolbarpalette id="BrowserToolbarPalette"> 
<toolbarbutton id="picnik-button" 
type="menu-button" 
class="toolbarbutton-1" 
label="&picnik. label;" 
tooltiptext="&picnik.menu;" 
oncommand="picnikScreenGrab. grabVisible() ;"> 
<menupopup> 
<menuitem Label="&picnik.grab_visible;" 
oncommand="picnikScreenGrab. grabVisible() ; 
event.stopPropagation() ;"/> 
<menuitem label="&picnik.grab_full;" 
oncommand="picnikScreenGrab. grabFull () ; 
event.stopPropagation() ;"/> 
</menupopup> 
</toolbarbutton> 
</toolbarpalette> 


</overlay> 


namespace (linuxjournal). This makes name clashes easier to avoid. 

Notice the call to window.addEventListener. This ensures that 
our onLoad function is called when the overlay is loaded. In the 
case of the generated code, it creates a variable that we can use 
to access the string bundle. 


Picnik 

Now that you have a basic understanding of how extensions are created, 
let's move on to a real-world example. My employer, Bitnik, recently pub- 
lished an API for its Flash-based photo editor, Picnik (www.picnik.com). 
Bitnik’s initial plan was to make our service easy to integrate into third-party 
Web sites. However, the API also opened the doors to using extensions to 
achieve a level of integration that we couldn't get with Flash alone. 

My first goal was to add a simple context menu item to allow 
users to edit existing photos easily. Luckily, most of that code is already 
in the code generated by the extension wizard. See Listings 1 and 
2 for the full text of the two most important files (picnik.xul and 


contextmenu.js). | also created common.js to store some variables that 
will be shared between files as we add features. 

Most of the modifications occurred in the function showContextMenu. 
This function is called immediately before the context menu is actually 
shown to the user. This gives our extension a chance to modify the menu 
items on the fly. In our case, | wanted to show only the Edit in Picnik option 
when the user was actually right-clicking on an image. 

Firefox provides our function with a global variable (gContextMenu), 
which contains a wealth of information about what the user clicked on. 
For example, gContextMenu.onlmage is true when the user activated the 
menu on an image. First, showContextMenu gets a reference to the actu- 
al menu item via its ID of picnik-ctx-edit. Then, it hides the item if the user 
didn’t click on an image. Finally, the function saves the image URL so that 
the extension knows what image to load into Picnik if the user actually 
selects the menu item. 

When the user selects the menu item, the browser calls 
picnik.editImage. This function constructs a URL to pass to Picnik 
and then creates a new tab with that URL. The server at picnik.com 
first downloads the image and then responds with a page containing 
the actual Flash application and the image, ready for editing. 


Screenshots 

While browsing on the Mozilla Developer Center’s Web site, | encoun- 
tered something that | saw as a natural improvement for Picnik’s Firefox 
extension—the ability to take screenshots of complete Web pages. The 
first part of the puzzle is the new Canvas HTML element that provides 
a flexible 2-D drawing canvas for JavaScript. Canvas originally was 
envisioned as a way to produce dynamic graphics client side. 

Two additional Canvas functions make screenshots possible. The 
irst is the drawWindow function. drawWindow, as its name implies, 
enders an XUL window to the canvas. In our case, we'll use it to 
ender the Web page. The second important function is toDataURL, 
which allows a script to get an image of what's on the canvas. 

Typically, URLs reference an object on a remote server or in the local 
ilesystem. Data URLs store the actual object as part of the URL. This 
can be handy for embedding small graphics directly in CSS or HTML. 
This technique allows the browser to avoid another request to the Web 
server. In our case, we'll use it to get a PNG file of our canvas. 

As you can see in Listing 3, picnik.xul has been modified to add a 
pop-up menu to both the context menu and the Tools menu. An additional 
ile, screengrab.js (Listing 4, available on the Linux Journal FTP site—see 
Resources), contains the code for actually grabbing the screenshot. As with 
he context menu code, this file also has an onLoad. In this case, however, 
he function's job is to detect whether either Canvas or toDataURL is miss- 
ing. If so, it disables the screen grab functionality. This allows the extension 
o run on Firefox 1.5 without confusing error messages. 

The two functions grabFull and grabVisible set up the parameters 
o grab either the full page or the visible area, respectively. They leave 
he bulk of the work to the aptly named grab function. In order to 
imit the amount of data that needs to be uploaded, grab scales the 
canvas so that it will be smaller than 2800x2800 before actually 
endering the window. Next, grab creates a canvas and renders 
he window before retrieving a data URL via toDataURL. 

saveDataUrl has the job of actually sending the image to Picnik. It 
constructs a multipart/form-data request containing several API parame- 
ers as well as the image data. Picnik’s servers respond with a URL for the 
browser to load. When that happens, requestState is called with a 
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req.readyState of 4. Finally, requestState creates a new tab with that URL. 


Distribution 
Most Firefox extensions are distributed at Mozilla's Add-ons site as 
Cross-Platform Installs (.xpi). This site provides users with a central 
trusted source for extensions. Because the site is pre-trusted by the 
browser, it makes installation easier for users. The Add-ons site also 
makes it easy to provide auto-updates to users. You simply can upload 
a new version, and users automatically will be prompted to upgrade. 
The extension wizard created a shell script (build.sh) to automate 
the process of creating the XPI. Once it's created, you can send it 
to friends, post it on your blog, upload it to addons.mozilla.org 
or distribute it any way you see fit. 


Conclusion 

| hope this article has given you an idea of how easy it is to write 
Firefox extensions and how powerful they can be. Don’t forget that 
you can learn by using the Chrome List extension to explore the inner 
workings of both the browser and other extensions. Also, the Mozilla 
Developer Center has a wealth of how-tos and reference resources 
waiting to be tapped. Happy coding! m 


Justin Huff is a Software Engineer with an embedded systems background, working for a little 
Web 2.x company called Bitnik. He resides in Seattle, Washington. 


Resources 


Code for this article, including screengrab.js (Listing 4): 
ftp.linuxjournal.com/pub/Ij/listings/issue160/9730.tgz 


Extension Developers Extension: ted.mielczarek.org/code/mozilla/ 
extensiondev 


Console?: https://addons.mozilla.org/en-US/firefox/addon/1815 
Chrome List: https://addons.mozilla.org/en-US/firefox/addon/4453 
Venkman: https://addons.mozilla.org/en-US/firefox/addon/216 
Firefox Extension Wizard: ted.mielczarek.org/code/mozilla/extensionwiz 
Mozilla Developer Center: developer.mozilla.org 

Mozilla Add-ons: addons.mozilla.org 


Picnik API: www.picnik.com/info/api 
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Streaming Audio with Ices 


and Icecast 


Retransmit from a radio scanner to the Internet via Ices and Icecast. BRIAN MATHERLY 


The Sioux Empire Amateur Radio Club operates a repeater on 
146.895MHz with the call name WOZWY. On Tuesday evenings, it 
provides a time for club announcements. | like to listen in when | can, 
but sometimes | am still at work when the announcements start and 
can't get to my radio. | found a way to solve this problem by using 
a UHF/VHF scanner, Linux, Ices and Icecast. By rebroadcasting the 
transmission over the Internet, | can listen to the local club repeater 
anytime, anywhere. 


Tuning In 
The first step was to find a VHF tuner that could tune to 146.895MHz 
and output analog audio to a sound card. | found that the least expen- 
sive way to go was to buy a consumer-grade UHF/VHF scanner. The 
scanner needed to have either a line-level output or a headphone jack. 
| found that line-level outputs are rare, but there are a number of scan- 
ners with headphone jacks. | also wanted one with a digital tuner so 
that it would not drift off frequency over time. | found a Radio Shack 
PRO-2050 on eBay for around $75 US that met all my requirements. 
Using an F-to-BNC adapter, | connected the scanner’s antenna jack 
to the off-air TV antenna on the roof of my house. This worked well 
because 146.895Mz lies right between off-air TV channels 6 and 7. | 
spent some time setting the squelch as high as | could, so there would 
not be static when no one was transmitting. Figure 1 shows my 
scanner with the audio cable plugged in to the headphone jack. 


Getting Connected 

In order to put the audio on the Internet, it first must be encoded. 
Then, that encoded information can be made available for streaming 
to clients on the Internet. For these tasks, Icecast and Ices make a 
great team. Icecast is a media streaming program that supports Ogg 


\MERT 192.166.1221 
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Figure 1. Radio Shack Scanner with Audio Cable Attached 
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Vorbis or MP3 streams. It receives encoded media from one or more 
sources and makes it available for streaming to multiple clients. Ices is 
an Ogg Vorbis audio encoder that works well with Icecast. | chose 
Ogg because it is a patent- and royalty-free format. Plenty of players 
can decode Ogg. XMMS on Linux and Winamp on Windows are the 
most common. If you are interested in using MP3 encoding, check out 
Livelce by following the link in the Resources for this article. 

The encoding and streaming tasks can be done on the same 
machine, but | chose to do them on separate machines. This makes it 
easier to add more sources later. Additionally, it removes some of the 
processing burden from the streaming machine. 

For the encoding computer, | chose a 233MHz Pentium | computer 
that had been unused for many years. | connected the headphone jack 
from the scanner to the line-in jack on the sound card using a 3.5mm- 
to-3.5mm audio cable that came from an old set of computer 
speakers. | connected the network interface to my LAN and set 
the IP address to 192.168.1.21. 

For the streaming computer, | chose to use my existing router, 
which doubles as a Web server. The main reasons | chose it are that | 
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Figure 2. Network Diagram 


already own it and it has my only public IP address. This computer has 
two network interfaces. One is connected to my cable modem and 
has a public IP address. The other interface is connected to my LAN 
and has the private IP address 192.168.1.1. Another way to accom- 
plish this would be to use a consumer-grade cable/DSL router and 
forward port 8000 to the streaming computer's IP address. 

Figure 2 shows a diagram of my network, including the scanner, 
encoder, streamer and cable modem. 


Icecast Configuration 

My streaming computer is running Mandriva. Icecast is available from 
Mandriva’s contrib RPM repository. Once | added that repository, 
installing Icecast was as easy as typing urpmi icecast from the com- 
mand prompt. If you want to install from source, you can download 
the latest release from the Icecast Web site (see Resources). Simply 
click the download link, extract the archive, and run the familiar 
commands ./configure; make; make install. 

The Icecast configuration file is an XML file and usually resides in 
/etc/icecast.xml. | was able to use most of the default settings, but | 
highlight some of the changes | did make here. 

The limits section of the file allows you to set the maximum num- 
ber of source streams (encoders) and the maximum number of clients 
that can connect to the streaming computer at once. The upload data 
rate on my cable modem is limited to 256Kbps by my ISP. As such, | 
need to limit the maximum number of clients to be sure that | don’t 
use all of my upload bandwidth: 


<limits> 
<clients>10</clients> 
<sources>2</sources> 


</lLimits> 


The authentication section is where you specify the user names 
and passwords. The source-password is the password used by the 
encoding machine when it connects. The relay-password is used by 
relays, which | am not using in my configuration. The admin-user and 
admin-password allow access to the administration Web page. It is 
important to change all passwords from the default for security: 


<authentication> 
<source-password>hackme</source-password> 
<relay-password>hackme</relay-password> 
<admin-user>admin</admin-user> 
<admin-password>hackme</admin-password> 


</authentication> 


The hostname is used so that Icecast knows what address to 
append to the beginning of the links on the Web page: 


<hostname>example.com</hostname> 


The listen-socket allows you to set the port on which Icecast 
listens; 8000 is the default: 


<listen-socket> 
<port>8000</port> 
</listen-socket> 


Once the configuration file is all ready, you can start Icecast by 
running the init script: /etc/init.d/icecast start. If there are any 
errors during startup, look in the log files to debug them. 


Ices Configuration 

My encoding computer is also running Mandriva. Because this 
machine is running only at 233MHz with 64MB of RAM, | decided 
to do a minimal install and leave off the window manager. Ices is 
also available from the contrib repository. So once again, typing 
urpmi ices is all it took to install Ices. However, you can down- 
load the latest releases from the Ices Web site (see Resources). 
Make sure you use the 2.0 series if you want to use Ogg. It also 
requires that libshout is installed. Once you have that, you can 
extract the archive and run ./configure; make; make install 
to get everything installed. 

The Ices configuration file is also an XML file. It is usually stored 
in /etc/ices.conf. 

The beginning of the configuration file has some settings that 
dictate how Ices runs. It can be helpful to change these during 
the initial setup, so that Ices runs in the foreground and sends 
messages to the console. These messages can be very helpful in 
debugging problems on initial setup. Once everything is set up 
and running, make sure Ices runs in the background and logs 
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or MP3 streams. 


messages to a file: 


<background>1</background> 
<logpath>/var/log/ices</logpath> 
<logfile>ices.log</logfile> 
<loglevel>3</loglevel> 


<consolelog>0</consolelog> 


The rest of the file is under the stream section. This is where 
you configure settings specific to this particular audio stream. 
Within the stream section, the metadata section is where you 
specify information about the stream. This information will be 
displayed on the Icecast Web page: 


<metadata> 
<name>WOZWY 146.895 MHz</name> 
<genre>Live</genre> 
<description>Live feed of the 
WOZWY repeater</description> 
</metadata> 


The input section is the place to define where the audio actually 
comes from. There are many possibilities, including options for playlists 


Ices Settings and Bitrates 


The quality, sample rate and channels settings all will affect the bitrate 
of the stream. Table 1 shows various combinations of these settings 
and their resulting bitrates. 


Table 1. Ices Settings and Bitrates 


Low-Quality 20Kbps 


Talk Radio 

Medium-Quality 2 11127 1 30Kbps 
Talk Radio 

Low-Quality 0 44100 2 56Kbps 
Music 

Medium-Quality 2 44100 2 84Kbps 
Music 

High-Quality 45 44100 2 132Kbps 


Music 
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and scripts. Because | want to encode live audio, | used the oss 
module. Don’t be alarmed if your system uses the ALSA sound system 
instead of OSS. ALSA has OSS compatibility, so this module works 
with both ALSA and OSS. Use the device parameter to specify the 
sound device from which to get data. On most systems it will be 
/dev/dsp. The rate parameter specifies the sample rate of the data in 
hertz. Most devices use 44100. Use the channels parameter to specify 
the number of channels available for capture. For most devices this 
will be 2 (stereo): 


<input> 
<module>oss</module> 
<param name='device'>/dev/dsp</param> 
<param name='rate'>44100</param> 
<param name='channels'>2</param> 


</input> 


The instance section allows you to specify the number of instances 
of this stream. You might have more than one instance if you want to 
send the stream to more than one server, or if you want to have dif- 
ferent versions of the same stream at different bitrates. For my system, 
| want only one instance. 

The first part of the instance section is where you specify the 
streaming server information. The hostname tells Ices where to send 
the data. The port and password must match the values you specified 
in the Icecast configuration file. The mount option specifies what the 
name of the stream will be called. 

The encode section specifies how the audio will be encoded. The 
easy way to do it is to set the sample rate and channels to match the 
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Figure 3. Icecast Status Page 


input section above. But, | didn’t need that much quality for my 
stream. So, | used the downmix and resample sections to tell Ices to 
resample the audio to 11127Hz and downmix it to 1 channel (mono). 
There are two options you can use to adjust the final bitrate of the 
stream: quality and nominal-bitrate. Notice that | commented out 
nominal-bitrate and set quality to 2: 


<instance> 
<hostname>192.168.1.1</hostname> 
<port>8000</port> 
<password>hackme</password> 


<mount>/146.895.ogg</mount> 


<encode> 
<quality>2</quality> 
<!--nominal-bitrate>32000</nominal-bitrate--> 
<samplerate>11127</samplerate> 
<channels>1</channels> 


</encode> 
<downmix>1</downmix> 


<resample> 
<in-rate>44100</in-rate> 
<out-rate>11127</out-rate> 
</resample> 


</instance> 


Once the configuration file is all ready, make sure that Ices has 
permission to access the audio device through /dev/dsp. Mandriva 
creates an audio group, which owns the /dev/dsp file. It also 
creates an ices user and group when ices is installed. | simply 
added ices as a member of the audio group in the /etc/groups 
file by editing the audio group: 


audio:x:81:ices 


Finally, you can start Ices by running the init script: 
/etc/init.d/ices start. If there are any errors during startup, look 
in the log files to debug them. It also can be helpful to examine the 
Icecast log files on your streaming computer to debug problems. 

Once everything is up and running, you can access the Icecast 
status page on the port you specified in the configuration file. Figure 3 
shows an example of an Icecast status page. Clicking on Click to 
Listen launches your audio player. 


Fine-Tuning 

Once | had everything set up and running, | spent a fair amount of 
time fine-tuning the audio. The first thing | found was that the audio 
level was quite low. Normally, when | want to turn up the audio level, 
| use KMix. But, | hadn't installed a window manager on my encoding 
computer. My system uses ALSA as the sound system, and | found 
out that there is a great curses-based mixer for ALSA named 
Alsamixer. It allows you to use the arrow keys to change settings. 
Figure 4 shows the settings | chose for my card. My final preference 
was to have the Master and PCM both at 100%. | also chose to mute 
everything else, because | knew they would not be in use. 


Figure 4. Alsamixer Settings 


Now that the audio was loud enough, | still had another problem. 
The audio sounded choppy. After looking around for a while, | realized 
that the processor usage was at 100%. | tried to reduce the bitrate by 
changing the value of the nominal-bitrate setting. With the bitrate set 
sufficiently low, the audio sounded good. | guess you can’t encode 
very high-quality audio with a 233MHz processor. 

When the audio sounded good, my goal was to lower the bitrate 
as much as possible while keeping an acceptable level of audio quality. 
In my case, an “acceptable level” is quite low. The traffic on the 
WOZWY repeater is only voice, because it is illegal to transmit music in 
the amateur radio bands. 

Three aspects of the encoded audio will affect its quality and 
bitrate: number of channels, sample rate and bitrate. | started by 
setting the downmix to 1, because the scanner is only mono. Then, 
| incrementally decreased the sample rate until | found the lowest 
setting that rendered acceptable audio; 11127Hz ended up being the 
minimum. Instead of setting the bitrate with the nominal-bitrate setting, 
| used the quality setting. The quality setting directly affects the final 
bitrate. With a few tries, | chose a quality level of 2. 

With my final settings, the bitrate stays around 30Kbps, and the 
processor hovers around 40% usage. 


Conclusion 

There are probably many other applications where this type of system 
would be useful. Keep in mind, however, if you are interested in using 
his type of system for streaming music, make sure you have permis- 
sion from the artists and recording studios. 

This combination of open-source projects really worked well 
ogether. Setting up everything was quite simple. | spent most of my 
ime tweaking the quality and sample rate settings. The traffic on my 
streamer is quite low. | usually have only one or two connections per 
week. But, it was fun to learn, and it is my little way of participating 
in the amateur radio community. 


Brian Matherly is a Software Engineer at Sencore Electronics in Sioux Falls, South Dakota. He is 
also an adjunct professor at Colorado Technical University. 


Resources 


Icecast Web Site: www.icecast.org 
Ices Web Site: www.icecast.org/ices.php 


Livelce Web Site: web.arm.ac.uk/~spm/software/liveice.html 
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Standard Operating 
Procedures for Embedded 


Linux Systems 


Follow these procedures for the smoothest path to great embedded Linux. 
CHI-HUNG CHOU, TSUNG-HSIEN YANG, SHIH-CHIANG TSAO AND YING-DAR LIN 


Procedures for developing embedded systems are very complicated. 
New engineers typically take a long time to become familiar with these 
procedures. Therefore, we have developed a standard operating proce- 
dure (SOP) to save the costs of constructing an embedded system and 
reduce the complexity. The SOP includes five standard procedures for 
building a Linux-based embedded system, as shown in Figure 1. You can 
follow the procedures discussed in this article for building a prototypal 
system. Also, we introduce ten useful methods for downsizing your 
system. Finally, we show the effect of these methods on downsizing 
your embedded system—a content-aware network security gateway. 


Five Standard Procedures 
To build an embedded system, the first step is to select a target platform. 
The platform involves both hardware and software. The hardware platform 
includes the processor, bus and I/O; the software platform includes the 
bootloader, kernel and root filesystem. You must select each item in the 
target platform carefully to ensure that the hardware and software work 
together. For instance, bootloaders relate directly to the hardware. If the 
selected bootloader does not support your hardware platform, the whole 
embedded system cannot power on. Moreover, an operating system that 
requires MMU may fail to collaborate with MMU-supported processors. 
Second, in addition to the target platform, a development platform 
also is necessary. You cannot compile embedded software programs on 
the target platform, because the target platform often has a small RAM 
and slow CPU to minimize cost and power consumption. Therefore, 
you need to prepare a development platform with a fast CPU and large 
RAM to compile these programs. Besides, because the two platforms 


5 standard procedures for building a system 
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Figure 1. Procedures and Methods for Building and Downsizing Your 
Embedded Systems 
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have different hardware architectures, a cross-compiler environment is 
necessary. Buildroot is such a package to offer this environment. It has 
a friendly user interface to assist in choosing the hardware platform 
and the required software package. By using Buildroot, you can gener- 
ate a cross-compilation toolchain and a root filesystem easily with 
built-in application packages for your embedded system. 

After setting up the environment, the next step is identifying the 
packages required by your system. You can accomplish this by selecting 
the built-in packages directly from the menuconfig of Buildroot, or you 
can download them from the Internet. In fact, Buildroot provides a list 
of useful packages, such as iproute2, freeswan and squid. Buildroot 
also ensures that these packages can link successfully with uClibc, a C 
library with a smaller size than Glibc. If you cannot find the suitable 
packages, you will have to modify existing packages or write new ones. 

Having obtained the required packages, the next step is integrating 
them into the embedded system. Integrating here means using a cross- 
compiler to compile the source code into forms that can be executed in 
the target platform, and then adding them into the root filesystem. You 
can add packages into the root filesystem through Buildroot in any of 
three ways, as shown in Figure 2. Method 1 is to select them directly from 
the options in Buildroot. If the packages are not available in Buildroot, you 
may need to write a makefile for the package to indicate how to down- 
load, configure, compile and install the package. Also, you need to modify 
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Figure 2. Three Methods for Adding Packages into Your Root Filesystem 


the config.in file of Buildroot to display the option of the package in the 
configuration menu. However, if you would not like to write these config- 
uration files, you can use Method 2. In Method 2, you simply place the 
compiled packages in the directory named customize, and then Buildroot 
copies these compiled packages into the root filesystem during the build- 
ing procedures, according to the rules given in customize.mk. However, if 
you simply want to verify the functionality of a single package, you don’t 
need to rebuild the whole image. The steps in Method 3 are to mount the 
root filesystem on any one directory and then copy the compiled packages 
into the directory. Finally, unmount the directory, and you will get an 
updated root image. However, Method 3 may fail if the free space in the 
mounted filesystem is not enough for the new packages. In that case, you 
can adjust the parameters given in ext2root.mk to reserve more free space 
in your root filesystem during the period of system development. 

Finally, Figure 2 depicts two ways to test and verify whether the func- 
tions of a package are normal. The basic way is to download the root 
filesystem into the target platform and execute the package directly. 
However, doing this usually takes a long time. Another way is to boot the 
root filesystem in a virtual machine, such as QEMU and VMware. Using a 
virtual machine to examine a compiled image is fast and convenient, but a 
virtual machine may not simulate some characteristics, such as hardware 
interrupts. Hardware interrupts involve quick reaction behavior, so they can- 
not be implemented by virtual machines easily. Finally, if the target platform 
has the same CPU architecture as your develop- 


step in downsizing the kernel. If you choose an inappropriate kernel, 
the system may be not only large but also unable to use processor 
power effectively. For example, a standard Linux kernel on a hardware 
platform without MMU cannot work normally. Such a hardware plat- 
form requires a specific MMU-less kernel, such as uClinux. Most peo- 
ple use the standard Linux kernel and attempt to trim its size. 

The next step is to include only the necessary modules in the stan- 
dard Linux kernel by a correct configuration. In fact, the default con- 
figuration of a Linux kernel includes many unused modules, which 
causes you to have a big kernel. Figure 3(a) shows the experimental 
results on the downsizing effect of the correct configuration. In this 
case, a system supporting TCP/IP has a kernel image that is only 
59.84% of the size of a system supporting all network protocols. 

To downsize the kernel, the third step is to use the optimization 
parameters when compiling the kernel. Using parameters -O1, -O2 
or -O3 can improve performance, and using -Os can reduce size. 
However, optimizing for both performance and size simultaneously is 
not possible. Therefore, we generally select -O2 to achieve a balance 
between size and performance. As shown in Figure 3(b), the -Os 
parameter reduces the size of the kernel image by 22.82% as 
compared with -O3, but it causes worse performance. 

Besides including only the necessary modules and compiling the ker- 
nel with the optimal parameters, to downsize the kernel further, you can 


ment platform, you can use chroot to replace 
your local system with the target root filesystem. 

By following the five procedures outlined, 
you can build the root filesystem for your 
embedded system. However, there is still one 
problem that may be troubling you—how to 
downsize your embedded systems or how to 
use less Flash RAM to store the kernel and root 
filesystem. Requiring less RAM means that you 
can cut the cost of your embedded system. 


Ten Downsizing Methods 

The organization of the downsizing issue is dis- 
played in the bottom of Figure 1. We divide the 
methods into two parts, because the software 
platform of an embedded system typically con- 
sists of a kernel and root filesystem. The first 
part is how to get a small kernel, and the sec- 
ond part is how to downsize each component 
in the root filesystem, including libraries and 
shells. The second part also discusses how to 
compress the whole root filesystem. We 
describe all methods in detail below, along with 
experimental results. After explaining all meth- 
ods, we show the effect of these methods on 
our laboratory embedded system, called the 
Wall system. Table 1 presents the specification 
for the Wall system. The system is a network 
security gateway that provides application-layer 
content filters, such as antispam and antivirus. 


Methods for the Linux Kernel 
Selecting an appropriate kernel is the first 
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Table 1. Specification of Wall 
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DNS relay, Dynamic DNS, 
Link load balance, Bridge mode 
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decrease the size of the static buffer and array allocated in the kernel, 
because the kernel typically declares a large buffer and array for standard 
PCs. To find out which buffer or array occupies large memory space, you 
can use the command nm. This command can list the allocated size of 
each variable in an object file. With that information, you can browse the 
corresponding source code of the object file and alter the initial size of the 
buffer or array. Another approach for shrinking the buffer size is to modify 
the options in the menuconfig of the kernel to decrease the maximum 
number of supported peripherals, as shown in Figure 3(c) and (d). 


Methods for the Root Filesystem 

As shown in Figure 1, we identify six methods for downsizing the root 
filesystem. First, you can adopt a tool called BusyBox, which provides a 
fairly complete environment for any small or embedded system. BusyBox 
combines tiny versions of many common UNIX utilities into a single 
small executable file, and it is highly modular, allowing commands to be 
included or excluded at compile time. The space used for BusyBox is 
7.04% of that of the original tool, as demonstrated in Figure 4. 

Next, we introduce three methods for removing unused libraries or 
downsizing required libraries. First, you can use the command Idd to 
identify the required shared libraries for each program, and then with 
this information, you can remove the unused libraries. Notably, if a 
shared library is not used by programs, you additionally should check 
whether it is used by other shared libraries. Figure 4 shows that remov- 
ing redundant libraries reduces the root filesystem to 6.55% of its origi- 
nal size. Second, you can replace the standard C library with a small C 
library, such as uClibc, Newlib or diet libc. Such libraries remove the 
unused functions, so their size is smaller than Glibc, as shown in Table 
2. This table presents the differences in functionality between the four 
libraries. Third, you can use a library optimizer tool named Libopt to 
rebuild the libraries that include the only necessary functions for the 
executable programs and shared libraries found in the root filesystem. 
This tool utilizes objdump and nm to gather information about library 
object files, shared libraries and executable programs. 

The fifth method for downsizing the root filesystem is to remove 
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Table 2. Comparison between Different C Libraries 


Small Smallest Small 


Largest 


Good Good Bad Normal 
Fastest Fast Fast Fast 
Yes Yes Yes 
Yes Yes Yes 
LGPL LGPL BSD, GPL 
menuconfig only make /configure 
Standard C Needs Often linked Managed by 
library cross-compiler as static library Red Hat 
toolchain 


unnecessary documents. You can eliminate some directories, such as 
/home, /mnt, /opt, /root, /boot and /proc, if unused. You also can remove 
the man, info, include and example directories to reduce the size when 
additionally integrating a package into the root filesystem. In general, an 
embedded system executes only specific programs, so users can operate 
it easily without the help documents or examples in these directories. 

The final method is to avoid uncompressing the whole root filesys- 
tem into SDRAM. The root filesystem is compressed to save the stored 
space, for example, Flash RAM. However, after the filesystem is uncom- 
pressed into SDRAM, the Flash memory allocated for the filesystem is 
no longer necessary. For instance, if the compressed size of the root 
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Figure 3. Effects of the Downsizing Methods on the Kernel 
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Figure 4. Effects of Downsizing Methods on the Root Filesystem 
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lem, you can use CRAMFS. CRAMIS is a read- 
only filesystem, designed for simplicity and 
space efficiency. You do not need to uncom- 


sbin 
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press a CRAMFS image before mounting it. A 
CRAMFS image is zlib-compressed, one page 
at a time to enable random read access. The 
metadata is not compressed, but is expressed 
in a terse representation that is more space-efficient than in traditional 
filesystems, such as ext2 or FAT. However, due to the read-only property 
of compressed files, random write access is hard to implement for 
them. As shown in Figure 4, CRAMFS compresses the filesystem to 
12.77% of its original size. 

Now that we've covered the six methods, let’s move on to the 
effect of these methods on the Wall Project, as shown in Figure 5. 
First, we used BusyBox to substitute for the multiple utility programs 
used in the original shell. Then, we compiled all the required packages 
with the parameters --strip-unneeded and -O2. Next, we used the 
commands strip and objcopy to remove the unnecessary contents of 
packages. Finally, we deleted unnecessary directories, such as man, 
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Figure 5. Downsizing Results on the Root Filesystem of the Wall Project 


info and example. Figure 5(a) illustrates the result of these processes. 
However, the size of Wall was still 139MB. Hence, we had to view the 
contents of /usr indepth, as shown in Figure 5(b) and (c). In the Wall 
Project, removing unneeded documents and files saved 20.6MB of 
space. About 15.9MB of space then can be saved by eliminating 
unused libraries. However, as you can see, Perl occupied much space 
in our system. Other methods may exist to solve this problem, but it is 
sufficient to consider only what we have done above. 

We found that the optimization of package size is also useful for 
downsizing when integrating a new package into the root filesystem. 
Actually, most programs and libraries are compiled at optimizing level 2 by 
default (gcc options -g and -O2) and are compiled for a specific CPU. On 
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errs 


Intel platforms, software is compiled for i386 processors by default. To 
minimize the package size, you should not adopt the -g option, which 
adds the debug info in the execution files. Additionally, remember to use 
-strip and --strip-all to remove all symbols. In more-advanced methods, we 
used the command readelf to check for any redundant sections in the 
execution files, and we used objcopy to remove those redundant sections. 
However, this approach may be not efficient for small programs. 


Conclusion 

This article describes the five procedures for making a Linux-based 
embedded system and describes ten methods for downsizing the ker- 
nel and the root filesystem. After we used these methods, our Wall 
Project was downsized by 26.18%. The experiment's results reveal that 
the two most efficient methods are giving correct kernel compilation 
parameters and using simplified tools and libraries in the root filesys- 
tem. Hopefully, this article helps you understand the procedures and 
problems when building a Linux-based embedded system.m 
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AlphaMail Is 
Scalable and 


Accessible 
Web Mail 


AlphaMail takes a unique approach to 
providing a Web-based IMAP client. 
TONY KAY 


AlphaMail is a high-performance, feature-rich, open-source 
Web mail system created at the University of Oregon. The inter- 
face includes message snippets in indexes, UTF8 composition and 
numerous viewers for attachments (such as image icon preview 
and file listings from tarballs). It also tries to strike a balance 
between desirable features and too much interface noise. It 
was created to address several problems that exist with other 
open-source and commercial Web mail systems. 


Performance 

The first concern AlphaMail addresses is performance. Almost all 
Web mail systems (such as Horde’s IMP Web mail Client and 
SquirrelMail) use IMAP from within the Web server, which is 
incapable of persisting an IMAP session. 

The IMAP protocol is designed to optimize access through 
persistent access, so this is an inherent and recognized prob- 
lem. The problem is usually mitigated with an IMAP proxy that 
maintains a persistent connection. The problems with this 
solution are multifaceted. 

One problem is that the code in the Web mail client itself 
cannot depend on the state of the IMAP connection and must 
repeat commands as if each mouse click were a new IMAP 
session. This is a problem, because the sequence of required 
events for a new session in the IMAP protocol include authen- 
ticating and selecting the desired folder. The benchmarks of 
several IMAP servers indicate that the repetition of the folder 
selection command, even if the folder is already authenticated 
and selected (that is, through a proxy), can cause significant 
extra server load. 

These inefficiencies could be addressed through improve- 
ments in the IMAP server and proxy algorithms, but another 
problem is intractable: a proxy cannot improve the protocol. 
The fact that the Web mail client is using IMAP forces it to 
behave as a complete standalone client. If the developers 
want to add a complex feature, such as conversation views 
(a la Google mail), which requires complex message cross- 
referencing across several folders, the protocol itself becomes 
a major impediment. 


AlphaMail solves these problems by including a middleware 
layer that uses a simplified and extensible protocol for the Web 
application and is responsible for optimizing access to the mail 
servers. The protocol supports highly specialized commands that 
allow the Web code to ask for the information needed for a page 
directly, without having to make any assumptions about the state 
of the IMAP connection. 

This has the additional advantage that the IMAP protocol handling 
in the middleware layer can be written in a high-performance 
language (in this case C++), can cache results and can optimize 
the interaction. The middleware program is known as the 
imap_webcache, because it caches both data and network 
connections for the mail interaction. 


Accessibility 

The next concern was accessibility. Information systems in higher 
education, government and many other environments must sup- 
port access for everyone. AlphaMail still is actively being tuned for 
access by the disabled, but it is already optimized for other access 
concerns. For example, site security policies might require or rec- 
ommend that members of the community disable browser features 
that regularly appear in US-CERT advisories, such as JavaScript. 
The only Web e-mail | could find that did not require JavaScript 
for even basic functionality was SquirrelMail, and it was deemed 
too risky from a performance standpoint. 

AlphaMail includes JavaScript enhancements, but those enhance- 
ments have traditional CGI alternatives that become active if JavaScript 
is disabled on the browser. All critical functions of the system will work 
with any browser, independent of capabilities and settings. 


Installation 
AlphaMail supports GNU autoconf and has been built cleanly on many 
Linux variants, FreeBSD, Solaris and Darwin (OS X). The easiest 
platform to install is Fedora Core 5 via yum(1), as described on the 
AlphaMail home page. 

| highly recommend using Fedora Core 5 on a test machine to 
avoid the headaches of dependency resolution for your initial trial run. 
The actual build is pretty much what you'd expect, but solving runtime 
dependencies can be challenging. 

If you decide on a source build, first you need to install the Boost 
C++ libraries (see Resources). 


Installing Boost 

You need the Boost Jam utility (usually named bjam) as well as 
version 1.33 or better of the Boost C++ libraries. Jam is sort of 
a combination between GNU autoconf and make. Follow the 
instructions from the Boost Web site for details, but essentially, 
extract the files and run: 


# bjam install 
In rare circumstances, you may want to pass options (such as an 
installation prefix). See the Getting Started guide on the Boost Web 


site if you have special needs. 


Building AlphaMail from Source 


The build is very much what you would expect: 


# ./configure 
# make 
# make install 


The configure script checks your system for dependencies and tells 
you what is missing or out of date. You should be able to complete all 
three steps as long as you have Boost installed, even if Perl dependen- 
cies are not met. 

Be aware that some versions of g++ have a bug that will cause the 
compiler to go into an infinite loop while building imap_webcache. 
Two of the files in the source (IMAPFolder_rules.cc and RFC2822.cc) 
can trip this bug, but even with a good compiler, these files take a 
large amount of time and space to build, so expect the compile to run 
for a few minutes. 

On some systems, there can be problems with the location of files. 
For example, on Red Hat Enterprise Linux systems, Kerberos depen- 
dencies cause OpenSSL code to fail to compile, which can be correct- 
ed by passing a value for CXXFLAGS: 


# CXXFLAGS="-I/usr/kerberos/include" ./configure 


As with any other GNU autoconf system, check the config.log if 
the configuration script fails to complete. 


Installing Runtime Dependencies 

The biggest difficulty on most distributions is not building 
AlphaMail, but rather meeting all of the prerequisites. Many 
distributions come with an older version of mod_perl, libapreq 
and other Perl modules. 

Make sure that you have libapreq2 and mod_perl >= 2.0 
installed before messing with the other Perl dependencies, 
because some of them rely on one or both. Once this is done, 
you should be able to install the remaining Perl dependencies with 
your package manager or cpan(1). 

You need to use cpan(1) if the packaged versions of a module do 
not exist or are too old. See the sidebar on avoiding packaging con- 
flicts with your distribution’s package manager. 


Configuration 

The build instructs you to create template configuration files with the 
alphamail_genconfig utility. This script prompts you for all of the nec- 
essary configuration options for a basic installation and creates the 
necessary files in a location you provide. 

You need a special user for a sandbox and knowledge of what 
user your Web server runs as before starting configuration. | recom- 
mend creating a user named sandbox for the former. Logins for this 
user should not be enabled. 

The configuration will ask for an installation prefix, which is what- 
ever you passed to configure. This is usually /usr/local, and the script 
will verify correctness before continuing. 

You will be asked to provide an IMAP prefix and separator for 
each IMAP server you want to access with AlphaMail. Some IMAP 
servers use slash (/) for the separator; others use dot (.). The prefix is a 
subfolder where users put all their other mail folders. For example, if 
users have shell access and their mail folders are stored in their home 
directories, it might be policy to put all of them in a directory named 
mail, in which case the prefix is probably mail. 
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Avoiding Package 
Management Conflicts 
When Using cpan(1) 


Many Linux administrators do not like to use cpan(1) due to 
competition between the files installed by the cpan utility and 
the distribution’s package manager. In production systems, 
therefore, it is desirable to install Perl modules in a place where 
the package manager will not see them. 


AlphaMail looks for its own libraries in /usr/local/lib/alphamail (if you 
chose the default prefix during the build), and you can install Perl 
dependencies there without having to change anything about the 
runtime configuration of the system. 


First, set your environment to indicate you would like Perl to look in 
an additional location (this is needed only during the build, the 
runtime system already includes this path during startup): 


# export PERLSLIB=/usr/local/lib/alphamail 


If you use an alternate prefix during configure, alter the /usr/local 
part of this to match. 


Next, run cpan(1) and (re)configure it: 


# Cpan 
cpan> o conf makepl_arg 
‘=> 'PREFIX=/tmp/unneeded LIB=/usr/local/1lib/alphamail' 


The PREFIX argument tells the build process what the general 
installation prefix is; whereas, LIB tells it where to install the actual 
module code. | use /tmp/unneeded for PREFIX and remove the files 
afterward, because AlphaMail needs only the library. Set PREFIX to 
something like /usr/local if you need the manual pages or other 
extras that come with these modules. 


If you want to save the cpan settings for future sessions, do: 
cpan> o conf commit 


Now, install or upgrade the necessary Perl modules (which are listed 
when you run configure): 


cpan> install Time: :HiRes 
lf you want to use these modules in your own scripts (or need 
to change where an AlphaMail script looks for them), add the 


following line near the top of the file: 


use Lib qw(/usr/local/lib/alphamail) ; 
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It is important to note that some Web servers (such as Cyrus) 
use INBOX for the prefix and dot (.) for the separator. The following 
procedure can help you determine what to use. First, connect to the 
IMAP server from the command line, with: 


# openssl s_client -connect imap.example.com:993 # For SSL 
or: 
# telnet imap.example.com 143 # for no SSL 


These commands connect you to the IMAP server and allow you to 
enter protocol commands. Type the following (the numbers are part of 
the commands): 


1 login username password 
2 list "" "qr 
3 logout 


The username and password, of course, should be real user 
credentials for a typical IMAP account. The responses to the second 
command should look like this: 


* LIST (\HasNoChildren) "." "INBOX.Spam" 
* LIST (\HasNoChildren) "." "INBOX.Trash" 


which indicates that . is the separator and makes it pretty 
obvious that INBOX is a common prefix (in this case all entries 
start with INBOX.). 

The prefix parameter is primarily an interface optimization: the 
interface removes the prefix when displaying most folder names in 
order to make things more compact. You can hand-edit any of the 
parameters in the resulting alphamail_config file, which is a 
commented text file. The entry for defining a pair of typical 
IMAP servers that serve two mail exchanges looks like this: 


imap_servers: example.com=imap.example.com:993[INBOX.], 


>example.net=imap.example.net:143[/] 


The above setting indicates that users should be able to select 
their mail domain on login (example.com or example.net), and associ- 
ates these with a corresponding IMAP server, port, prefix and IMAP 
path separator. 

The separator in the brackets is always required, but the prefix is 
not. The notation [/] means no prefix, with slash as the separator. The 
IMAP connections will be insecure if you use anything but the SSL 
alternate port 993. 

Attachment viewers and other external programs run in a sandbox 
that uses a chroot jail, user ID protections and other filesystem restric- 
tions to ensure that a bug in a viewer cannot compromise anything 
more than the file the user is trying to view, which by definition would 
be the file containing the exploit. This is where you will use the extra 
user you created earlier. 

The sandbox utility is installed in /usr/local/libexec/sandbox, by 
default, and is a setuid program. It is important that the permissions of 
this executable allow execution by the Web server, but it is a security 
hazard to allow any other user access to the utility. | recommend that 


AlphaMail be run on a standalone system that serves only Web mail 
and nothing else, with no shell access for users. 

The configuration also asks you to configure the large file-sharing 
system. This option allows users to upload files to the AlphaMail sys- 
tem, so that others can download them later. Large file sharing is use- 
ful when someone needs to send a file that is larger than is allowed or 
recommended as part of an e-mail message. File sharing has several 
safeguards to prevent abuse, including terms-of-use agreements, size 
limits, password protection, encryption, download limits and time- 
based expirations. Choosing a zero size for the size limit in file sharing 
disables the feature. 

The final step is to edit the Apache configuration. Make sure that 
mod_perl2 and libapreq2 are loaded with directives, such as: 


LoadModule apreq_module modules/mod_apreq2.so 


LoadModule perl_module modules/mod_perl.so 


And, include the generated alphamail.conf Apache configuration file. 
For example: 


Include /usr/local/etc/alphamail/apache/alphamail.conf 


Running AlphaMail 

Apache and imap_webcache must be running for AlphaMail to 
work. Startup order does not matter. A sample Red Hat init 
script for the Web cache is included and will be installed in 
/ust/local/share/alphamail/util/init.d. 

A garbage collection script must be run periodically from cron. 
AlphaMail writes numerous files as the mail system operates, most of 
which are decoded MIME messages and attachments. These files can- 
not be cleaned reliably by the Web software, as there are no guaran- 
tees about user behavior. The script is called garbage_sweeper and is 
well documented in the Administration Guide. 

AlphaMail is in production use at the University of Oregon. The 
performance and usability results have been very encouraging, and the 
former are available at the AlphaMail home page. 

However, the system is still new, and there are some latent bugs 
that have yet to be solved. The imap_webcache itself is a rather compli- 
cated piece of software that may have occasional problems. As a result, 
| recommend running an included utility called the hang_detector (in 
/usr/local/share/alphamail/util by default). You must edit this script 
before using it, and it requires a valid IMAP user in order to work. 

It runs a full query against the Web cache every 15 seconds and is 
capable of restarting the imap_webcache (via the included init script). 
It is also capable of sending mail to administrators if desired. 


Common Problems 

There are several places where you can run into difficulty on a new 
installation. The Administrator's Guide included with AlphaMail has 
many tips on how to solve these issues. 

One of the most puzzling problems is produced by accessing 
the service via an incorrect domain name. The cookie that main- 
tains the session is tied to the server domain that you specify 
during configuration and is stored in a parameter in the AlphaMail 
apache configuration: 


PerlSetVar alphamailDomain server.example.com 


lf the user is able to access the login page with an unqualified 
hostname (such as server), the cookie will not be exchanged properly 
and login will fail without any error whatsoever. The login is actually 
succeeding, but the browser is not sending the cookie back because 
of the mismatched URL. 

Fortunately, most users connect though a link or type an insecure 
HTTP URL. The former is never a problem, and a redirect does a nice 
job of correcting the latter: 


<VirtualHost _default_:80> 
RedirectMatch “/alphamail/?$ https://server.example.com/ 
> alphamail/index.html 
RedirectMatch */alphamail/mail/?$ https://server.example.com/ 
alphamail/mail/index.html 
</VirtualHost> 


Other problems usually involve dependencies, configuration 
errors, incorrect permissions or missing auxiliary directories. The 
best indication of these are errors in alphamail_ui.log, which by 
default is created in /var/log. 

SELinux also can be the source of problems. | have not run 
AlphaMail with SELinux, and unless you are willing to create 
your own security profile, you probably will need to turn it off 
or disable enforcing. 


The Future of AlphaMail 

AlphaMail development has been mostly concerned with perfor- 
mance and core functionality to this point, and | don’t expect 
these two issues to become any less important in the future, as 
the product is directed at environments that have a large and 
diverse user community. 

It is quite feature-rich, but certainly not all-inclusive. Limiting fea- 
ture creep is an ongoing requirement, because the critical concern is 
easy and reliable access to essential functionality, not global coverage 
of mail client features. 

Still, there are plans for significant improvements and additions, 
such as an internationalized interface, optional Ajax components that 
would improve client interaction and conversation views. 


Tony Kay is the primary developer of AlphaMail and works for the University of Oregon in Eugene, 
Oregon. He can he reached at tkay@uoregon.edu or tony.kay@gmail.com. 


Resources 


AlphaMail Administration Guide: sourceforge.net/docman/ 
display_doc.php?docid=38047&group_id=183361 


AlphaMail Home Page: sourceforge.net/projects/alphamail 
Boost C++ Libraries: www.boost.org 


IMAP Performance Benchmarks: 
whizzo.uoregon.edu/public/src/mailperf/results.html 


US CERT: www.us-cert.gov 
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The Benevolent Racketeer 


We present a fair and balanced picture of the other side of the Microsoft covenant not to sue. 


Nick Petreley, Editor in Chief 


We at Linux Journal have taken a dim view 
of the Microsoft/Novell deal and Microsoft's 
attempts to monetize the work of the Open 
Source community by offering covenants not 
to sue. However, in an effort to remain fair 
and balanced in our coverage of these con- 
troversial issues, we are publishing the fol- 
lowing testimonial from a happy Microsoft 
customer who wishes to remain anonymous 
because he doesn’t exist. 

Dear Linux Journal Editor, 

| must take issue with the negative stance 
Linux Journal has adopted with respect to 
Microsoft software patents, licensing and 
covenants not to sue. Our company represents 
one of the customers to whom Microsoft 
Senior Vice President Bob Muglia refers when 
he says, “Customers have indicated to us that 
it's problematic to them that when they work 
with open-source software, they don’t have 
similar intellectual property protection.” 

| find this to be so true. Our company 
simply could not adopt open-source software 
without intellectual property protection. Open 
source represented a grave liability to us 
because no company claimed control over it. 
Put simply, how could we have confidence 
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that someone would not sue us for using 
Linux unless we paid someone not to sue? 
We were anxious to pay protection money to 
solve this problem, but to whom would we 
pay it? We would gladly have paid Red Hat, 
but Red Hat did not threaten to sue us, so 
what would be the point? That's the problem 
with Linux. It is not owned by a single entity 
with the power to license protection, and 
those who distribute it are remiss in threaten- 
ing to sue in order to make intellectual prop- 
erty protection available to paying customers. 
We feel Linux advocates and developers 
should be grateful that Microsoft stepped up 
to the plate in this regard. Now that 
Microsoft has graciously offered to solve this 
dilemma for us by threatening to sue us for 
using Linux, we have finally found a reason 
to adopt Linux and open source. We now 
know where to send our protection money. 
Technically speaking, SCO was the first to 
offer us a covenant not to sue, and we were 
almost ready to adopt Linux in a big way 
when SCO provided us this option. We 
backed away when IBM and Novell tried to 
make it difficult for SCO to follow through. 
Why? IBM is much bigger than SCO. If IBM 
created enough trouble to put SCO out of 
business, it would set us back to square one, 
without anyone to pay protection money. 
This was a risk we were not willing to accept. 
Then Microsoft approached us behind the 
scenes and offered us a sweet deal. We 
agreed to pay Microsoft $20 for every copy of 
Linux we would install, in return for which 
Microsoft promised not to sue us for $400 per 
copy throughout the terms of the agreement. 
This deal not only provided us with the intel- 
lectual property protection we customers 
demand, as noted by Bob Muglia, it also saves 
us $380 per copy of Linux until the agreement 
expires! Think of it. Even if we deploy only 
1,000 copies of Linux, that’s a $380,000 
savings. With an economic incentive like that, 


how can anyone question Microsoft’s commit- 
ment to the open-source model? 

This is not the first time Microsoft volun- 
teered to assist us with intellectual property 
protection. Some years ago, Microsoft con- 
ducted an audit of our Windows and Office 
licenses, which brought to our attention the 
fact that we had not organized and stored 
our product licenses properly. Microsoft 
solved our problem easily by selling us all the 
replacement licenses we needed. It may 
seem unreasonable to some people to pay 
twice for the same software, but we beg to 
differ. We think of it as paying not for soft- 
ware, but for peace of mind, which is what 
intellectual property protection is all about. 

Some like-minded customers advised us 
to adopt SUSE Linux, as it is covered under 
the Microsoft/Novell covenant not to sue. 
We consider this an acceptable option, but 
not the ideal one. It is much more effective 
to deal directly with the threatening party. 

Think of it this way. Our regular fire 
insurance covers accidental fires. We've had 
two accidental warehouse fires (minor ones, 
thankfully) since we bought the policy. In 
what way did our insurance protect us 
against accidental fires? It didn't help at all. 
In contrast, we donate money in a manila 
envelope each month to another “insurance” 
company who has sworn to protect us 
against arson. Since we started making those 
payments, we've seen a 100% reduction in 
fires caused by arson. You can’t argue with 
that kind of success, and it’s all because we 
deal directly with the source of arson fires. 

| hope this testimony has opened your 
eyes at Linux Journal. It’s not too late 
for you to secure your own intellectual 
property protection. m™ 


Nicholas Petreley is Editor in Chief of Linux Journal and a former 
programmer, teacher, analyst and consultant who has been 
working with and writing about Linux for more than ten years. 


“Fanatical Support” saved me 
from my own mistake.” 


“Not long ago, | reformatted one of our servers. Not until | was driving home did | learn that | brought our entire 
site down in the process. | called my guy at Rackspace and he said, ‘We're already on it.’ By the time | pulled 
in the driveway, my site was back up. Now that's Fanatical Support." 


Keeping little mistakes from causing big problems is one definition of Fanatical Support. What will yours be? 


Watch Russ’s story at www.rackspace.com/fanatical 
1-888-571-8976 
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MANAGED BHOSTING 


Hearsyvourselfathink¢Again! 


WhisperStation” 


Cool!) Fast] Silent! 


For 64-bit HPC, Gaming and Graphic Design Applications 

Originally designed for a group of power hungry, demanding engineers in the automotive industry, 
WhisperStation™ incorporates two dual core AMD Opteron™ or Intel@ EM64T™ processors, ultra-quiet 
fans and power supplies, plus internal sound-proofing that produce a powerful, but silent, computational 
platform. The WhisperStation™ comes standard with 2 GB high speed memory, an NVIDIA e-GeForce 
or Quadro PCI Express graphics adapter, and 20" LCD display. It can be configured to your exact 
hardware specification with any Linux distribution. RAID is also available. WhisperStation™ will also 
make a system administrator very happy, when used as a master node for a Microway cluster! 

Visit www.microway.com for more technical information. 


@=Microway 


ON Technology you can count on” 


